PlanetSquires Forums

Support Forums => WinFBE - Code Editor and Visual Designer => Topic started by: dcouzin on December 18, 2018, 10:30:06 PM

Title: fumbling first questions
Post by: dcouzin on December 18, 2018, 10:30:06 PM
Hello Forum: I turned to FreeBASIC just two days ago, when my move to 64-bit Windows meant no more NTVDM to run my old QuickBASIC programs.  They're optical design programs written 25 years ago in QB version 3.00, and still useful. What a delight to find the programs could run 5 times faster using the FB compiler instead of QB's, and no more NTVDM and BRUN. 

My two goals:
1. to run the programs as fast as possible.
2. to not have to significantly rewrite the programs.
Do the goals conflict?
 
Paul Squires' latest suite with FreeBASIC-1.06.0 and WinFBE-1.8.7 worked straightaway.  The '-lang qb' option allowed most of my programs to compile after modest changes.  In QB I could name a variable 'X' and also name an array 'X()'.  I could express ≥ interchangeably with '>=' and '=>'.  Etc.  Etc. FreeBASIC-1.06.0 freaks out at these liberties and its error messages barely hint what's wrong.  Traces of guilt helped me to find them.

Without speaking for real programmers, I learned much of QuickBASIC v.3.00 syntax by trial and error.  Whatever the compiler allowed, I kept doing.  With time I should learn the new syntax allowed by the '-lang qb' option, but I fear that the new compiler will react to some deeper abuses/liberties in my programs that I won't fathom.

First question: do the earlier FreeBASIC compilers have a more tolerant '-lang qb' option than later FreeBASIC compilers?  Has the FreeBASIC principle of supporting QuickBASIC coding softened over time?

Second question: will the later FreeBASIC compilers provide a speed advantage for my programs?  The programs are just piles of arithmetic and trigonometry, running on a normal Lenovo T480s with Windows 10.

Besides those general questions, I'm mystified by specifics in the FreeBASIC-1.06.0 and WinFBE-1.8.7 combination.  One of my programs compiles and runs fine with build configuration 'Win32 GUI (Release)' and option '-lang qb', but with build configuration 'Win64 GUI (Release)' and option '-lang qb' it compiles only to crash after 2 seconds running.  How can this happen?

Will this forum suit someone who programs, but is not a programmer?

Dennis
Title: Re: fumbling first questions
Post by: Klaas Holland on December 19, 2018, 04:48:25 AM
Yes Dennis, this forum is a good place for you, because Paul and Jose are experts in helping you.

I'am not a programmer too and did switch from QuickBasic --> PowerBasic --> FireFly/PB --> FireFly/FB,
and I'am still using the same basics.
Title: Re: fumbling first questions
Post by: Paul Squires on December 19, 2018, 08:57:00 AM
Hello Forum: I turned to FreeBASIC just two days ago, when my move to 64-bit Windows meant no more NTVDM to run my old QuickBASIC programs.  They're optical design programs written 25 years ago in QB version 3.00, and still useful. What a delight to find the programs could run 5 times faster using the FB compiler instead of QB's, and no more NTVDM and BRUN. 
Hi Dennis - welcome aboard! :)

Quote
My two goals:
1. to run the programs as fast as possible.
2. to not have to significantly rewrite the programs.
Do the goals conflict?
As you've noticed, you have had to do some tweaking of your programs. I NEVER use '-lang qb' ever and have dropped any of my old QuickBasic habits long ago (after I switched to PowerBasic). Personally, I would try compiling using '-lang fb' as that is the future of the compiler. Legacy QB compatibility should be frozen at FB 1.05 and stripped out of the compiler for versions after that(but that's my opinion and not widely shared by the greater FB community who have a lot of old QB code). If you ever decide to start using WinFBE's visual designer then it will only create code using -lang fb.
 
Quote
First question: do the earlier FreeBASIC compilers have a more tolerant '-lang qb' option than later FreeBASIC compilers?  Has the FreeBASIC principle of supporting QuickBASIC coding softened over time?
If anything, QB support got better with the compilers as time went on. Bugs fixed and quirks in the syntax fixed. My "feeling" is QB support will remain in the compiler but development in the FB syntax has taken over and always be the main area of future development.

Quote
Second question: will the later FreeBASIC compilers provide a speed advantage for my programs?  The programs are just piles of arithmetic and trigonometry, running on a normal Lenovo T480s with Windows 10.
Most likely they will especially the 64 bit versions that uses GCC for the compiling backend. Whenever GCC is updated we normally see speed improvements and optimizations. There is an experimental LLVM backend that one day may prove to be even a little faster.

Quote
Besides those general questions, I'm mystified by specifics in the FreeBASIC-1.06.0 and WinFBE-1.8.7 combination.  One of my programs compiles and runs fine with build configuration 'Win32 GUI (Release)' and option '-lang qb', but with build configuration 'Win64 GUI (Release)' and option '-lang qb' it compiles only to crash after 2 seconds running.  How can this happen?
32 bit code and 64 bit code can sometimes be a problematic. The most damning area would be if you use the INTEGER variable (I always use LONG just to avoid issues). An INTEGER is 32 bit on 32 bit systems, but is 64 bits on 64 bit operating systems.

Quote
Will this forum suit someone who programs, but is not a programmer?
Definitely. I venture to say that the vast majority of us here are not professional programmers. :-)
Title: Re: fumbling first questions
Post by: Johan Klassen on December 19, 2018, 10:29:59 AM
also, if you decide to convert your QB code to FB, be mindful of the integer type differences
QB Integer ( number% ) – Integer variables are 2 bytes long, FB32 Integer is 4 bytes long - FB64 Integer is 8 bytes long
QB Long Integer ( number& ) -Long Integer variables are 4 bytes long
in FB you can declare an integer like so
Dim as Integer n ' 4 or 8 bytes long
or
Dim as Integer<16> n '2 bytes long, valid sizes are 8, 16, 32 and 64 bits
Title: Re: fumbling first questions
Post by: raymw on December 19, 2018, 09:28:52 PM
Hi Dennis,

It depends if it is worth the effort for you, but I would consider trying to work out what your largest numbers are likely to be, and chose integers that will just about hold them. e.g, if your largest value is going to be in the hundreds range, then no point in doing calculations with 64 bit integers. If rounding errors are unacceptable, do not use floating point numbers, and so forth. Take this as an opportunity to rewrite your code in freebasic, as Paul suggested. If it is software you developed yourself, and you have the documentation ;-) it may be as easy to rewrite it anyway, from your original specification, instead of trying to convert your existing qb code line for line.

Best wishes,

Ray
Title: Re: fumbling first questions
Post by: dcouzin on December 20, 2018, 10:36:21 AM
Dear all:
Is the only reason for using the 64-bit compiler large integer calculation?  But why would the 64-bit compilation crash when the integers happen to be small?  My program declares some integer variables; none take values over 300.  Why should the FreeBASIC 64-bit compilation not execute when the FreeBASIC 32-bit compilation and the QuickBASIC 16-bit compilation do?  Shouldn't the crash concern the developers?  In the crash scenario, the fat DOS cursor blinks for about two seconds and then the window disappears.

After reading a 2008 discussion at freebasic.net/forum I appreciate how QuickBASIC needed reform, and FreeBASIC fulfilled that.  GOSUB leads to spaghetti code.  It's fun to write, but just a few months later it's hard to untangle.  Now, 25 years later and faced with a calculation core with 79 GOSUB's, 27 RETURN's, 72 GOTO's in 1400 lines, untangling is beyond my powers.

The code was originally written in the assembly-like language of the TI-95, and then transformed to QuickBASIC.  Besides those two encodes there's little documentation.  If the '-lang qb' option now allows compilation of my old code, why rewrite it in FB?  As an exercise for learning FB?  It's too difficult an exercise.  Better to learn a new language through writing small new programs.  The compiler signals a few errors, you correct those, and you're a step farther into the new language.   

I added FreeBASIC-1.05.0 to the WinFBE_Suite directory to compare with FreeBASIC-1.06.0.  (That a newbie can do this testifies to the clarity of WinFBE.)  For that 1400 line program, the 1.06.0 compiler made a 78,848 byte executable, compared to the 1.05.0 compiler's 76,288 byte executable.  What did the 2,560 bytes of extra lard in the first compilation provide?  Not faster execution.  The 1.05.0 version is actually 0.5% faster -- based on many runs, it's a real difference. 

In the film "Orphee" (1949) Cocteau gives his definition of "poet", (Englished): "One who writes, but is not a writer."  Can we find the nice X defined: "one who programs, but is not a programmer"?

Dennis
Title: Re: fumbling first questions
Post by: Johan Klassen on December 20, 2018, 12:03:08 PM
@dcouzin
the reason I suggested to be mindful of the difference in integer size is that a lot of FreeBASIC programs that were written for the 32-bit version fail when compiled with the 64-bit version because of the assumption on the integer size being 32-bit
but since you are not inclined to learn to program in FreeBASIC but only want to compile and run your old Qbasic programs then I suggest you try QB64 http://www.qb64.org
QB64 prides itself in Qbasic compatibility, though the executables are bloated and usually run twice as slow as that produced by FreeBASIC
QB64 forum https://www.qb64.org/forum/index.php
Title: Re: fumbling first questions
Post by: dcouzin on December 20, 2018, 04:51:39 PM
@Johan Klassen
Maybe I now understand what went wrong with my integer variables.  It was the '-lang qb' option that forced the FB 64-bit compiler to make the integer variables 32-bit.  Then the execution failed.  Why didn't the 64-bit compiler "know" that execution had to fail with 32-bit integers?  Why, instead of reporting errors, did the compiler pass off a doomed executable?  Perhaps I rely too much on error reporting by the FB compiler.  If ever the FB 32-bit compiler produced an executable that just crashed, I wouldn't know what to do.

My first post stated my two goals:
1. to run the programs as fast as possible.
2. to not have to significantly rewrite the programs.

Since the FreeBASIC 32-bit compiler, with option '-lang qb', achieves the goals, why would I ever choose QB64, which you say is slow, over FreeBASIC?  Well, I just now tried QB64.  It accepted my QuickBASIC program as-is: pleasant.  But its executable ran 2.5◊ slower than FreeBASIC's: disqualifying for most of my work.  FreeBASIC's executable uses 24% of my CPU compared to 11% for QB64's.

I see no advantage, yet, to 64-bit compilation.  My programs don't need over 2 GB of memory.  If fact the tested FreeBASIC executable used just 7 MB of memory.  I certainly don't need integers larger than 2^32 -1.  What other advantages are claimed by 64-bit compilations?

Dennis

Title: Re: fumbling first questions
Post by: Josť Roca on December 20, 2018, 05:28:48 PM
The main advantage of a 64-bit compiler is to address more memory. If you have enough with 32-bit, then use it. As simple as that.
Title: Re: fumbling first questions
Post by: Johan Klassen on December 20, 2018, 08:50:52 PM
@dcouzin
I just downloaded the latest stable version of QB64 for Windows and run a simple speed test and it's slow, the test speed was 12+ second
I then downloaded the 64-bit version from http://qb64.freeforums.net/thread/100/qb64-x64-10-17-2018 and the test speed was about 0.5 seconds
hope this helps you :-)
Title: Re: fumbling first questions
Post by: Johan Klassen on December 20, 2018, 10:01:36 PM
@dcouzin
in my view, QB64 may be useful to you, if you get the 64-bit version linked in the post above
and there's a trick to get your program to run as fast as one compiled with FB
first place this meta-command in the top of your program
Code: [Select]
$CHECKING:OFF
do a test-run, it's so that QB64 will create the file recompile_win.bat in the directory qb64/internal/temp/
open that file and right after g++ add a space and -O3, save and run the file to recompile your program
Title: Re: fumbling first questions
Post by: dcouzin on December 21, 2018, 01:55:59 AM
@Johann Klassen
I tried what you suggested with qb64-x64.  With the first compilation my test program took 70 seconds to run.  After the recompilation it took 54 seconds.  FreeBASIC takes 30 seconds.
My sample code must be very different from yours, to benefit so much less from the recompilation.

Dennis
Title: Re: fumbling first questions
Post by: Johan Klassen on December 21, 2018, 05:58:46 AM
@dcouzin
perhaps g++ reuses the object files from the unoptimized compile if the source file has not changed, may I suggest one more try
open your program in the editor and enter a character somewhere and then delete that character and save the file, exit the editor
reopen recompile_win.bat and place -O3 as before and recompile, hope that will do the trick.
 
Title: Re: fumbling first questions
Post by: Johan Klassen on December 21, 2018, 06:28:59 AM
@dcouzin
since execution speed is important and FreeBASIC32 is compiling you program ok, I suggest you compile your program with compiler option -gen gcc -Wc -Ofast
sometimes gcc can make good optimazations
Title: Re: fumbling first questions
Post by: dcouzin on December 21, 2018, 09:50:32 PM
@Johann Klassen
I couldn't make your qb64-x64 trick work.  The recompilation is always the same.  (It is smaller than the first compilation and gives 54 second execution instead of 70 second.) 

How could fiddling with pgm.bas effect the "object files" connected with pgm.exe anyhow?  I assume you mean the many files in the directory qb64/internal/temp/ that were created when pgm.exe was created.  I also tried deleting some of those files, but then recompile_win.bat wouldn't work.  Perhaps there are certain ones I can delete in order to get a different recompile.

On the other hand, your suggestion for the FreeBASIC compiler worked beautifully.
Making the compiler options: '-lang qb -gen gcc -Wc -Ofast' the compilation came out smaller (61,952 bytes vs. 78,848 bytes) and the execution time dropped from 30 seconds to 16 seconds.  Super!

The '-gen gcc' option worked with FreeBASIC-1.06.0\fbc32.exe, but not with FreeBASIC-1.05.0\fbc32.exe.  Is that to be expected?

Dennis
Title: Re: fumbling first questions
Post by: Johan Klassen on December 22, 2018, 11:10:23 PM
@dcouzin
I am glad that the suggestion for FreeBASIC worked for you, as for -gen gcc not working with version 1.05.0, I suspect that gcc may be missing in your distribution.
Title: Re: fumbling first questions
Post by: dcouzin on December 25, 2018, 02:28:45 PM
@Johann Klassen
FreeBASIC-1.06.0\bin\win32\ contains gcc.exe.  gcc.exe is nowhere inside FreeBASIC-1.05.0 or FreeBASIC-1.04.0, which each have gprof.exe (which FreeBASIC-1.06.0 doesn't have).
Are there compiler options for FreeBASIC-1.05.0 or FreeBASIC-1.04.0 corresponding to the one you suggested for FreeBASIC-1.06.0?

Dennis
Title: Re: fumbling first questions
Post by: Paul Squires on December 25, 2018, 03:44:32 PM
Why use 1.04 or 1.05 just to save a few KB and a maybe a second or two of execution? I would much rather use the latest compiler to ensure that I am compiling with the most bug free version of the compiler.
Title: Re: fumbling first questions
Post by: Johan Klassen on December 25, 2018, 06:12:41 PM
@dcouzin
I recommend that you just copy the bin folder from FreeBASIC-1.06.0-win32 to FreeBASIC-1.05.0-win32 but first rename the bin folder in version 1.05.0 to something else, that way you can revert if it doesn't work.
beware that versions before 1.06.0 may have bugs that since have been fixed, and perhaps the C emitter may not be as good as that of version 1.06.0
I have not investigated if there are optimization options using the default -gen gas for 32-bit FB, I am inclined to think that if they are present, would not be as good as that of gcc

[edit] better use the FreeBASIC-1.06.0-win32 bin folder from the version available here https://www.freebasic.net/forum/viewtopic.php?p=248057#p248057
Title: Re: fumbling first questions
Post by: dcouzin on December 26, 2018, 09:21:18 AM
@Paul Squires
As the FreeBASIC language evolves the compiler must evolve, always with new bugs and their fixes.  But I am not yet a fan of FreeBASIC.  Parts of me prefer the less fussy syntax of QuickBASIC.  So at this time, for running FB with the '-lang qb' option, it's a different question whether 1.04.0, 1.05.0, or 1.06.0 is least buggy than it is for the '-lang fb' option.  The farther the language evolves away from QB, the easier for a new fix to break the old '-lang qb' option. 

So far, all three FB versions are giving correct answers for the QB code I'm feeding them.  Now with the 'gen gcc' option there are no speed differences, just the size difference of the executable.  I should use whichever version is least likely to have a lurking bug for my code.

Dennis
Title: Re: fumbling first questions
Post by: Paul Squires on December 26, 2018, 09:34:52 AM
Here is the link to the changelog so you can see what has changed/fixed over 1.04, 1.05, 1.06. I wouldn't be so much concerned about any -lang qb breakages as I would about fixes and optimizations that later compiler versions are giving to the -gen gcc output. More recent versions have better -gen gcc output.
https://github.com/freebasic/fbc/blob/master/changelog.txt

I hope that you can get all of the versions working and that FB serves you well with your program. I totally understand and appreciate your situation with your legacy code.
Title: Re: fumbling first questions
Post by: dcouzin on December 26, 2018, 09:38:55 AM
... I recommend that you just copy the bin folder from FreeBASIC-1.06.0-win32 to FreeBASIC-1.05.0-win32 ...

This worked.  Is this a hybrid of 1.06.0 and 1.05.0?  The 'gen gcc' option then works, and execution time matches that with 1.06.0.  The executable filesize is a bit smaller, as with old 1.05.0.

...[edit] better use the FreeBASIC-1.06.0-win32 bin folder from the version available here https://www.freebasic.net/forum/viewtopic.php?p=248057#p248057

[modified 1 hr after first posting] The WinFBE_Suite I'm using was released 6 months after that post at freebasic.net, and it uses the preferred 8.1.0 gcc.

Dennis
Title: Re: fumbling first questions
Post by: dcouzin on December 26, 2018, 11:08:10 AM
... I hope that you can get all of the versions working and that FB serves you well with your program. I totally understand and appreciate your situation with your legacy code.

Thanks.  I actually have about 500 QB87 (QuickBASIC v.3.00) programs, 200 of which still butter my consulting bread.  All 200 are based on the same calculational core: 1400 lines of numerical operations.  Fifteen numerical input parameters set the core running to yield one number.  Each of the 200 programs twiddles the input parameters differently and runs the core up to millions of times.  Each twiddling code is transparent, mostly loops, and I could probably write them in FB.  But the core is opaque to me now, so I must just test/trust that FB with '-lang qb' produces correct numbers from the core.

Dennis 
Title: Re: fumbling first questions
Post by: Johan Klassen on December 26, 2018, 04:46:16 PM
@dcouzin
I personally only use the -Ofast option for trivial programs, -Ofast or -O3 may optimize your program to the point where it gives wrong results
in your case, since you use your programs for serious applications, I would either use -O2 or -Os
-Os is equal to -O2 with an emphasis on optimizing for small code
Title: Re: fumbling first questions
Post by: dcouzin on December 26, 2018, 11:18:23 PM
... -Ofast or -O3 may optimize your program to the point where it gives wrong results
in your case, since you use your programs for serious applications, I would either use -O2 or -Os ...

According to this advice, I can have no option '-gen gcc' benefit at all.
As I originally used FB, with options simply '-lang qb', the test program ran in 29.5 sec.
Then with options '-lang qb -gen gcc -Wc -Ofast' the test program ran in 15.5 sec, a nice improvement.
But replacing '-Ofast' with either '-Os' or '-O2' or '-O3' makes the test program run in about 39 sec.

It boggles my mind that there can be over-optimizations that cause seriously wrong calculations.  What exactly does '-Ofast' do?  I know the kinds of calculation in my core program and might be able to judge whether it's immune to the dangers of '-Ofast'.

The 5 different optimizations do cause extremely slight differences in the calculated output.  The '-Os', '-O2', and 'O3' optimizations give identical results to the 15th displayed digit.  The optimization with simply '-lang qb' agrees with them through 14 digits.  The optimization with '-Ofast' agrees only through 13 digits.  These differences are zilch to me, who only needs 3 or 4 good significant digits, though I'm curious where '-Ofast' chisels.

Incidentally, when I run the test program in QB64, the calculated output agrees with the FB output through just 8 digits.  Using QuickBASIC (v.3.00) through NTDVM, my old standard, there's agreement with the FB output through just 6 digits.  I'd noticed that and it wasn't a concern.  My only concern is that a compiler might do something cuckoo.

Dennis
Title: Re: fumbling first questions
Post by: Johan Klassen on December 27, 2018, 08:20:43 AM
@dcouzin
I just wanted you to be aware that there could be a problem when using -Ofast, if you have enough tests to ensure that the results are always acceptable then you can make a decision whether you want to use it or not.
have a look at the gcc documentation on optimization https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html
Quote
-ffast-math
Sets the options -fno-math-errno, -funsafe-math-optimizations, -ffinite-math-only, -fno-rounding-math, -fno-signaling-nans, -fcx-limited-range and -fexcess-precision=fast.

This option causes the preprocessor macro __FAST_MATH__ to be defined.

This option is not turned on by any -O option besides -Ofast since it can result in incorrect output for programs that depend on an exact implementation of IEEE or ISO rules/specifications for math functions. It may, however, yield faster code for programs that do not require the guarantees of these specifications.
Title: Re: fumbling first questions
Post by: dcouzin on December 27, 2018, 12:47:12 PM
@Johan Klassen
Thanks much for the link to the gcc documentation.  I'm now warned about -Ofast, so will do an extensive test over all the input parameters.
What's puzzling now is that setting the options simply as '-lang qb' results in a faster running compilation than use of '-gen gcc' together with any of '-O2', '-O3', or 'Os'.  What kind of optimization does FreeBASIC 1.06.0 do when options are set simply as '-lang qb'?

Dennis
Title: Re: fumbling first questions
Post by: Johan Klassen on December 27, 2018, 01:12:01 PM
I am not familiar with the compiler internals to answer that question.
Title: Re: fumbling first questions
Post by: dcouzin on December 28, 2018, 10:13:34 AM
My reported speed trials were poorly done.  I made the mistake of doing repeated runs of the calculational core with identical parameters, only outputting the value computed last.  This would allow a clever compiler to skip all or part of the first many calculations.  It also ignored the time needed for writes to disk.  I've now redone the speed trial on a real version of my optical design program.  It writes almost 400,000 single precision values to disk, each one requiring (I think) 100 runs of the calculational core.   

The new speed results are surprising.  The real program, which used to take overnight to run in QB87 compilation with NTDVM, took these times to run with FB compilation, with various options set:
32.4 min with '-lang qb'
31.3 min with '-lang qb -gen gcc -Wc -Ofast'
51.8 min with '-lang qb -gen gcc -Wc -O2'
51.8 min with '-lang qb -gen gcc -Wc -Os'
51.8 min with '-lang qb -gen gcc -Wc -O3'

The single precision output values are identical for the different compilation options, so there is hardly reason to specify any gcc options for this program. 

The writes to disk are not taking significant time.  When I doubled the core runs to 200 (all different) required for each write the times just doubled.

FreeBASIC 1.06.0 compiles this program very well with the '-lang qb' option set.  Is it making use, in its own wise way, of gcc?

Dennis
Title: Re: fumbling first questions
Post by: Paul Squires on December 28, 2018, 10:24:35 AM
-lang qb is outputting using the assembler backend because -gen gas is the default output. The problem is that if you want 64 bit compiles then you need to use the -gen gcc parameter and invoke the GCC backend. In that case, FB compiles the code to C and passes that code to GCC to compile the 64 bit EXE.

I guess for certain scenarios the -gen gas is outperforming the -gen gcc.



Title: Re: fumbling first questions
Post by: Andrew Lindsay on January 23, 2019, 12:33:53 AM
Just a suggestion, if your code/program is not something that is trade secret, it may be beneficial to share it?  I know that I have had lots of help with the guys on this forum and at Powerbasic in helping to optimize code.
I've done a little bit of demangling too to convert from Gosub-Return to structured code.
And how knows, someone may have already solved the problem you're seeking a solution for.