PlanetSquires Forums

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1] 2

Author Topic: some basic filing stuff  (Read 3685 times)

raymw

  • Senior Member
  • ***
  • Posts: 363
some basic filing stuff
« on: August 12, 2019, 03:12:46 PM »

Just thinking about advantages/disadvantages of different formats. In particular strings being saved, e.g. name, age, date of birth/whatever. Comma separated values are quite common, and often can be used as input to spread sheets, but not if the strings may or may not have commas. Using print#1 a,b will space at 14 column boundaries padding with blanks (chr(32) ) as far as I know. This leads to large files, but it is nice to read in virtually any text editor. There are characters such as ASCII 30, 31 which are probably a hangover from earlier teletype days for unit and record separation, which saves some file space. Alternatively, since the items on a line will be in a known order, then each could be on a separate line, so to speak. But, I have a sort of gut feeling that if there was an error in saving /reading (or transmitting/receiving) the data, then the whole stream would be out of sync, whereas reading say three values on one line, the next lines would be ok, if one of those values was misread.
So, what have you done, and why did you chose that method.
Logged

Paul Squires

  • Administrator
  • Guru Member
  • *****
  • Posts: 8937
  • Windows 10
    • PlanetSquires Software
Re: some basic filing stuff
« Reply #1 on: August 12, 2019, 03:57:28 PM »

For CSV files with strings containing commas you simply need to wrap the entire string in double quotes (if I remember correctly).

Personally, for any application where I need a local database, I use SQLite. It's simple to use and allows me to invoke SQL commands that make data analysis easy.

If you still need to use a sequential text file then you can also use the TAB character (Chr(9)) as a separator value instead of a comma. Most spreadsheet programs recognize TABs as a field delimiter. Construct your string and then simply PRINT #f, st
Logged
Paul Squires
PlanetSquires Software
WinFBE Editor and Visual Designer

SeaVipe

  • Senior Member
  • ***
  • Posts: 244
  • Windows 10
Re: some basic filing stuff
« Reply #2 on: August 12, 2019, 04:42:25 PM »

Hi Ray,
Paul mentioned SQLite and I think that could work (if I understand your question correctly). I have tried several times to wrap my head around SQLite and even though the sample files are pretty simple and easy to follow, I always throw my hands up and revert back to that which I know, not that which I should be using.


My programs have far too many CSV, ini, text and binary files when they could all be handled (much) better if I would just take the time to learn SQLite.


There are too many lines of code like this in my programs:
Code: [Select]
Dim as Long o = Open( t_db.dBaseName For Random as #eFileNum.maindb Len = SizeOf( t_crm ) )


FYI: I spent the weekend building an XML parser; that might be a format you could consider.
@raymw

Logged
Clive Richey
There is nothing government can give you that it hasn't already taken from you in the first place. Winston Churchill

Paul Squires

  • Administrator
  • Guru Member
  • *****
  • Posts: 8937
  • Windows 10
    • PlanetSquires Software
Re: some basic filing stuff
« Reply #3 on: August 12, 2019, 05:48:18 PM »

Hi Clive, in the past I used XML for storage but I have found JSON to be easier to write and less "noisey" in the way the file looks.

Jose has SQLite FreeBasic classes in his WinFBX\Afx\CSqlite3. inc file. Very easy to use and once you understand SQL then even complex joins are easy (I model my SQL in the free, open source, program SQLiteStudio (portable edition) before I code the SQL string in my progams).
Logged
Paul Squires
PlanetSquires Software
WinFBE Editor and Visual Designer

SeaVipe

  • Senior Member
  • ***
  • Posts: 244
  • Windows 10
Re: some basic filing stuff
« Reply #4 on: August 12, 2019, 07:04:19 PM »

Thanks, Paul,
SQLiteStudio certainly looks interesting. Might have saved me a weekend of coding!


I'm looking into XML only because an Android app that backs up my Calls and SMS is stored in that format. My CRM app (still in FF3 and not yet converted to WinFBE) currently requires that I manually add Calls and SMS data to each client individually - although somewhat automated it is still very labour intensive and time consuming. I thought that if I can extract the data from the XML file it could be linked to my clients as needed for printing and filing purposes.


About to try SQLiteStudio and then Jose's classes - again.
Thanks again.
Logged
Clive Richey
There is nothing government can give you that it hasn't already taken from you in the first place. Winston Churchill

raymw

  • Senior Member
  • ***
  • Posts: 363
Re: some basic filing stuff
« Reply #5 on: August 13, 2019, 06:46:54 AM »

Thanks fr the suggestions/comments. I think Sqlite is a bit of an overkill for what I want, and I don't want to learn even more methodologies which I'm unlikely to need in future. I've a shrink-wrapped copy of Quatro-pro, still on the shelf, from back when I was doing this stuff for payment. I've decided to use tabs as separators, although many spreadsheets these days allow other separators to be used. Talking of spreadsheets and 'back then', I remember investigating a convoluted linked spreadsheet system, about 30 sheets, with which you could run a business - ordering stock, invoicing, statuary returns, all of it. I guess, if you have a hammer, everything looks like a nail.
Logged

raymw

  • Senior Member
  • ***
  • Posts: 363
Re: some basic filing stuff
« Reply #6 on: August 14, 2019, 10:28:35 AM »

Part of the data strings I'm interested in is date +time. It seems a bit odd, in that if you use Date (after #include "vbcompat.bi" ) it gives the date in month, day, year order. But it seems all calculations on dates require the order to be the regional operating system settings. Although the date formats can be changed, it would be more straightforward if the date function gave the resulting string as the regional order. Also, I've a feeling that the time/date functions should be combined - i.e. time to include the date. perhaps change the whole system to 16months of 16days (and 109 intercalary days) or maybe get the earth to whizz round the sun a bit faster.
Logged

Paul Squires

  • Administrator
  • Guru Member
  • *****
  • Posts: 8937
  • Windows 10
    • PlanetSquires Software
Re: some basic filing stuff
« Reply #7 on: August 14, 2019, 10:45:42 AM »

Part of the data strings I'm interested in is date +time. It seems a bit odd, in that if you use Date (after #include "vbcompat.bi" ) it gives the date in month, day, year order. But it seems all calculations on dates require the order to be the regional operating system settings. Although the date formats can be changed, it would be more straightforward if the date function gave the resulting string as the regional order. Also, I've a feeling that the time/date functions should be combined - i.e. time to include the date. perhaps change the whole system to 16months of 16days (and 109 intercalary days) or maybe get the earth to whizz round the sun a bit faster.

These are things that you would have to address with the FreeBASIC compiler team. They maintain that code.

Also, as I tell everyone who uses FB and WinFBE, you MUST download and use Jose's WinFBX libraries of code. There are THOUSANDS of extremely useful code functions there that will GREATLY simplify your coding life. For date and time functions check out his AfxTime.inc source file.
Logged
Paul Squires
PlanetSquires Software
WinFBE Editor and Visual Designer

raymw

  • Senior Member
  • ***
  • Posts: 363
Re: some basic filing stuff
« Reply #8 on: August 14, 2019, 06:05:15 PM »

Hi Paul, I've been playing with the afx time functions, but seems i can't get them all to work

Code: [Select]
#include"afx/afxtime.inc"

dim t as longlong
t=afxtime64()
?t
'? afxtime64tofiletime(t)
'? afxtime64tosystemtime(t)

sleep

although t is a valid long long, if the two lines are uncommented, a compiler error message says invalid data types, which I do not understand, since t should be valid, according to my understanding of the help
Logged

Josť Roca

  • Guru Member
  • *****
  • Posts: 3217
Re: some basic filing stuff
« Reply #9 on: August 14, 2019, 06:42:38 PM »

What is not valid is to use PRINT to display a FILETIME or SYSTEMTIME structure that is what these functions return.

Paul Squires

  • Administrator
  • Guru Member
  • *****
  • Posts: 8937
  • Windows 10
    • PlanetSquires Software
Re: some basic filing stuff
« Reply #10 on: August 14, 2019, 06:52:33 PM »

Not sure exactly what it is you're trying to do but the following code might help:

Code: [Select]
#include "afx/afxtime.inc"

'dim t as longlong
't = afxtime64()
'? t
'? afxtime64tofiletime(t)
'? afxtime64tosystemtime(t)

dim as ULONGLONG uTime = AfxQuadDateTime()
? AfxQuadDateToStr(  "ddd',' MMM dd yyy", uTime )
? AfxQuadTimeToStr( "hh':'mm':'ss tt", uTime )


sleep
Logged
Paul Squires
PlanetSquires Software
WinFBE Editor and Visual Designer

raymw

  • Senior Member
  • ***
  • Posts: 363
Re: some basic filing stuff
« Reply #11 on: August 14, 2019, 08:19:28 PM »

I have a file of lines of text with three strings on each line. The lines look like this
Code: [Select]
Garden 21.5 08-11-2019_18:53:06
Garden 22.0 08-11-2019_18:53:11
Greenhouse 23.0 08-11-2019_18:53:29
Garden 21.5 08-11-2019_18:53:29
Greenhouse 23.5 08-11-2019_18:53:32
Greenhouse 23.0 08-11-2019_18:53:44
Greenhouse 23.5 08-11-2019_18:53:47


basically, location, temperature, date/time of reading. I want to be able to manipulate the data, remove the 'jitter' in the temperature readings, plot out charts/graphs etc. My immediate quest is for a simple method to convert the date/time string to a number, so that I can step through the file, and plot out the average temperature for each location over a period of say one minute or more, and of course a simple way to convert the number back to a date_time string. A reading is added into the file when the temperature changes, the temperatures are scanned every two seconds, although that scan rate will be much slower when I've got it all working.
Logged

Paul Squires

  • Administrator
  • Guru Member
  • *****
  • Posts: 8937
  • Windows 10
    • PlanetSquires Software
Re: some basic filing stuff
« Reply #12 on: August 14, 2019, 08:30:48 PM »

In that case, simply append to each line the AfxQuadDateTime() value. That will save you from having to convert your mm-dd-yyyy_hh:mm:ss value that you are currently saving to the file. Once you have the AfxQuadDateTime() value you can easily convert it to a date and/or time in whatever format you wish similar to my example in the previous post.
Logged
Paul Squires
PlanetSquires Software
WinFBE Editor and Visual Designer

raymw

  • Senior Member
  • ***
  • Posts: 363
Re: some basic filing stuff
« Reply #13 on: August 15, 2019, 06:04:25 AM »

If I wanted to store extra time/date information, i think the most useful would be 'the time between readings'. If I started a second counter, storing the value in a 'long', it would be 68 years before I'd have to think about it again. Unless there is an incredible medical breakthrough, that will be an impossible scenario.
Logged

raymw

  • Senior Member
  • ***
  • Posts: 363
Re: some basic filing stuff
« Reply #14 on: August 16, 2019, 11:51:02 AM »

Hi Paul, iirc you had a few common dialogs in firefly. I thought there was a tutorial or something showing a similar file dialog for freebasic, but I may have imagined that. Will you be having an easy generation of common dialogs within winfbe?

OK I've pre-empted your reply - use Jose's AFx code.
« Last Edit: August 16, 2019, 01:10:29 PM by raymw »
Logged
Pages: [1] 2