- Welcome to PlanetSquires Forums.
PlanetSquires Forums
"It’s not a bug; it’s an undocumented feature" - Anonymous
Recent posts
#31
PlanetSquires Software / USTRING data type (built into ...
Last post by Paul Squires - August 06, 2026, 06:55:22 AMUSTRING: Universal String
A dynamic UTF-16 string type for FreeBASIC (Win/Linux, 32/64 bit)
Information and source found at the github repository: https://github.com/PaulSquires/USTRING
Full compiled FBC compiler toolchains for 32/64 Windows and Linux are included in the repository.
\fbc-win
\fbc-linux
USTRING is true intrinsic type, not a library.
USTRING is not a UDT, a class, a macro, or a header you #include. It is a data type inside the compiler, registered exactly the way ZSTRING and WSTRING are.
It replaces the need to use José's DWSTRING class in the AfxNova library.
The FB 1.20.0 source was directly modified so that USTRING becomes the 4th string data type in FB (along with STRING, ZSTRING, and WSTRING).
USTRING passes all FBC tests. To see the string in action, take a look at the ustring_usage_test.bas file located in the \tests subfolder. It showcases USTRING used every way the FB language allows. Every construct is checked, not just compiled, so it doubles as documentation you can trust.
Covered: every declaration form, dynamic and fixed, arrays (static, dynamic, multi-dimensional, initialised), UDTs, inheritance and virtual methods, properties, operator overloads, every parameter mode (BYVAL / BYREF / BYREF AS CONST, dynamic and fixed), pointers, and every return form (BYVAL, BYREF, fixed-length).
I hope to introduce USTRING to the FB maintainers for inclusion into the core FB compiler. It solves one of the most longstanding omissions in the FB core language - a dynamic unicode enabled string data type.
The core engine
Three things: a descriptor, an allocator, and hand-rolled primitives.
The pieces that matter:
ushort vs. extending wstring
WSTRING has two problems, and they're independent:
(1) it can't grow;
(2) its element width varies by platform — 2 bytes on Windows, 4 on Linux, 1 on DOS.
Extending WSTRING to be dynamic fixes (1) and leaves (2) — and (2) is the one that actually corrupts things. You'd end up with a dynamic string that's still UTF-16 on one target and UTF-32 on another: same source, different bytes on disk, different offsets, different LEN. It would also be a breaking change to a type whose sizeof and ABI programs already depend on.
So the element is uint16_t explicitly, never wchar_t — with a compile-time assert enforcing it. That's what makes the representation identical everywhere.
The subtle part is the no-libc rule, and it's the reason for spelling it uint16_t rather than "wchar_t where it happens to be 16 bits". On Windows they are the same width, so wcslen, wcsstr, towlower would link and appear to work perfectly. Using them there and hand-rolled code elsewhere is exactly how two platforms drift apart — which is the failure this type exists to remove. Same reasoning drove using a generated case table instead of towupper, and a fixed UTF-8 conversion instead of the C locale.
Why UTF-16 internally, not UTF-8
Deliberate trade:
Note this only concerns the in-memory form. Files are UTF-8 regardless, and STRING ↔ USTRING is UTF-8 on every target — so the UTF-16 choice never leaks into anything portable.
NEXT UP: Generics
Once Generics become a core FB language feature, it will be easy to have:
(1) Iterator protocol
(2) For Each: for each x in c
(3) Containers: Vector, Dictionary, Set, List
A dynamic UTF-16 string type for FreeBASIC (Win/Linux, 32/64 bit)
Information and source found at the github repository: https://github.com/PaulSquires/USTRING
Full compiled FBC compiler toolchains for 32/64 Windows and Linux are included in the repository.
\fbc-win
\fbc-linux
USTRING is true intrinsic type, not a library.
USTRING is not a UDT, a class, a macro, or a header you #include. It is a data type inside the compiler, registered exactly the way ZSTRING and WSTRING are.
It replaces the need to use José's DWSTRING class in the AfxNova library.
The FB 1.20.0 source was directly modified so that USTRING becomes the 4th string data type in FB (along with STRING, ZSTRING, and WSTRING).
USTRING passes all FBC tests. To see the string in action, take a look at the ustring_usage_test.bas file located in the \tests subfolder. It showcases USTRING used every way the FB language allows. Every construct is checked, not just compiled, so it doubles as documentation you can trust.
Covered: every declaration form, dynamic and fixed, arrays (static, dynamic, multi-dimensional, initialised), UDTs, inheritance and virtual methods, properties, operator overloads, every parameter mode (BYVAL / BYREF / BYREF AS CONST, dynamic and fixed), pointers, and every return form (BYVAL, BYREF, fixed-length).
I hope to introduce USTRING to the FB maintainers for inclusion into the core FB compiler. It solves one of the most longstanding omissions in the FB core language - a dynamic unicode enabled string data type.
The core engine
Three things: a descriptor, an allocator, and hand-rolled primitives.
Code Select
typedef struct _FBUSTRING {
FB_UCHAR *data; /* uint16_t, on every target */
ssize_t len; /* CODE UNITS, not bytes */
ssize_t size; /* allocated units, excl. terminator */
} FBUSTRING;
12 bytes on 32-bit, 24 on 64-bit — asserted at compile time against what symb-data.bas hardcodes, because if those disagree every ustring variable is the wrong size on the stack.The pieces that matter:
- Length is authoritative, the terminator is a courtesy. Every operation uses len, so embedded NULs survive and LEN is O(1). That's also why appending is O(1) amortised while WSTRING's is O(n²) — no walk to find the end.
- Size-discriminator ABI. Every entry point takes a (ptr, size) pair: -1 = var-len descriptor, size & FB_STRISFIXED = USTRING * N, 0 = raw units. Same scheme FBSTRING uses, so one function body serves all three forms.
- Temp-descriptor pool — 256 slots, identified by pointer range, with the temp flag in len's sign bit. Plus descriptor stealing, which is what keeps a = b + c + d linear instead of allocating per term.
- Geometric growth — (units + 15) & ~15 then +12.5% headroom (ustr_core.c:77).
- No libc wide functions. All primitives hand-written in ustr_prim.c.
ushort vs. extending wstring
WSTRING has two problems, and they're independent:
(1) it can't grow;
(2) its element width varies by platform — 2 bytes on Windows, 4 on Linux, 1 on DOS.
Extending WSTRING to be dynamic fixes (1) and leaves (2) — and (2) is the one that actually corrupts things. You'd end up with a dynamic string that's still UTF-16 on one target and UTF-32 on another: same source, different bytes on disk, different offsets, different LEN. It would also be a breaking change to a type whose sizeof and ABI programs already depend on.
So the element is uint16_t explicitly, never wchar_t — with a compile-time assert enforcing it. That's what makes the representation identical everywhere.
The subtle part is the no-libc rule, and it's the reason for spelling it uint16_t rather than "wchar_t where it happens to be 16 bits". On Windows they are the same width, so wcslen, wcsstr, towlower would link and appear to work perfectly. Using them there and hand-rolled code elsewhere is exactly how two platforms drift apart — which is the failure this type exists to remove. Same reasoning drove using a generated case table instead of towupper, and a fixed UTF-8 conversion instead of the C locale.
Why UTF-16 internally, not UTF-8
Deliberate trade:
- Bought: O(1) LEN and []; free Win32 ...W interop (a pointer reinterpret, no copy); semantics familiar from WSTRING-on-Windows.
- Paid: 2 bytes per character even for ASCII, and a surrogate pair counts as 2 for LEN — so MID can split one, exactly as on a WSTRING.
Note this only concerns the in-memory form. Files are UTF-8 regardless, and STRING ↔ USTRING is UTF-8 on every target — so the UTF-16 choice never leaks into anything portable.
NEXT UP: Generics
Once Generics become a core FB language feature, it will be easy to have:
(1) Iterator protocol
(2) For Each: for each x in c
(3) Containers: Vector, Dictionary, Set, List
#32
PlanetSquires Software / Re: Tiko - beta tests
Last post by Paul Squires - August 06, 2026, 05:33:21 AMExcellent, thanks Hans, appreciate it 
#33
PlanetSquires Software / Re: Tiko - beta tests
Last post by hajubu - August 06, 2026, 02:32:16 AMHi Paul,
Thanks for beta-Test phase - works fine -and here my remarks
a) Screen display on old Laptop 1366x768 :full sized windows: settings pop-up uses app.80% , Fonts resized to 10
b) Big 27inch - need only to be resized à gusto : visual acuity : sizing and local position
c) Observing a "crash" ::
Starting the Tiko out-of-the box , loading a test file and try to compile:
==> error ( warning pop-up --> cannot not find compiler path
Then it stops like a "crash" without further option for adapting new settings-
(... logically this might correct, but new user not knowing the background may be scared.)
Background : out-of-the-box path setting may be (are) not known and may be different from mine .
Tiko ought to be able to find it's way i.e. "default path" in the "Toolchains" below the own path.
P.S. My ideas are ::
::@(b) :: Output windows for Big Screen :: option for display in a "side panel"
::@(c) :: an stop-over on the settings page for that choice.
b.r. (Hajubu) Hans
Thanks for beta-Test phase - works fine -and here my remarks
a) Screen display on old Laptop 1366x768 :full sized windows: settings pop-up uses app.80% , Fonts resized to 10
b) Big 27inch - need only to be resized à gusto : visual acuity : sizing and local position
c) Observing a "crash" ::
Starting the Tiko out-of-the box , loading a test file and try to compile:
==> error ( warning pop-up --> cannot not find compiler path
Then it stops like a "crash" without further option for adapting new settings-
(... logically this might correct, but new user not knowing the background may be scared.)
Background : out-of-the-box path setting may be (are) not known and may be different from mine .
Tiko ought to be able to find it's way i.e. "default path" in the "Toolchains" below the own path.
P.S. My ideas are ::
::@(b) :: Output windows for Big Screen :: option for display in a "side panel"
::@(c) :: an stop-over on the settings page for that choice.
b.r. (Hajubu) Hans
#34
PlanetSquires Software / Re: Tiko - beta tests
Last post by Paul Squires - August 06, 2026, 12:33:52 AMNo major reports in the 3 days that this beta has been active.
I am taking that as a positive sign.
@José Roca if you have tried this beta then I would be interested to know if the dialog and font sizes are okay on your computer/laptop. This beta's dialogs and fonts are larger than in the most recent released version. I remember that you had trouble with the long height of top menus a version or two ago, so I expect that screen real estate is important for you.
I am taking that as a positive sign.
@José Roca if you have tried this beta then I would be interested to know if the dialog and font sizes are okay on your computer/laptop. This beta's dialogs and fonts are larger than in the most recent released version. I remember that you had trouble with the long height of top menus a version or two ago, so I expect that screen real estate is important for you.
#35
José Roca Software / Re: AfxComplex-&-AfxMath
Last post by hajubu - August 05, 2026, 04:36:08 PMThanks - worked - Super !
#36
José Roca Software / Re: AfxComplex-&-AfxMath
Last post by José Roca - August 05, 2026, 03:20:17 PM #37
José Roca Software / Re: AfxComplex-&-AfxMath
Last post by José Roca - August 05, 2026, 11:02:43 AMThanks. I will look at it. I have uploaded AfxMath.inc by mistake. It is a work in progress almost finished that I have began to document.
#38
José Roca Software / AfxComplex-&-AfxMath
Last post by hajubu - August 05, 2026, 09:13:07 AMCode Select
Hi, Jose
just dowloaded the latest AfxNova (AfxNova-260804_af18ed4) and got and error 4 Duplicate Definition.
It seams that we have in AfxComplex or/and AfxMath duplicates.
(AfxComplex line 87 & 297 ) // (AfxMath line 753 another FUNCTION IsInfinity Function )
taking the old AfxComplex runs fine even with AfxMath.inc aside.
May you have a closer look - Thanks
Hajubu
```
#console on
/' simple testcase with Tiko 260731 and also Tiko2-beta (_dev_260804_0324_445bbd84) '/
#include "AfxNova\CComplex.inc"
DIM z AS _complex = (1.2345, -2.3456)
print CAbsSqr(z)
SLEEP(5000)
----
AfxNova-260804_2208_af18ed4 ==> Duplicated definition
new upload AfxComplex.inc . AfxMath.inc
Line: 87 --- error 4: Duplicated definition in 'DECLARE FUNCTION IsInfinity (BYVAL x AS DOUBLE) AS LONG'
Line: 297 --- error 4: Duplicated definition in 'PRIVATE FUNCTION CComplex.IsInfinity (BYVAL x AS DOUBLE) AS LONG'
----
AfxNova-260801_1608_b4ab951 ==> runs without error
old AfxComplex.inc -BUT has had NO AfxMath.inc - as all before - ,
upload was only DWStrProcs.Inc
```
#39
PlanetSquires Software / Tiko - beta tests
Last post by Paul Squires - August 04, 2026, 11:54:29 AMHi everyone,
I know that some of you try the latest Tiko versions via the github "development" branch. I can inform you that the branch now contains the latest most feature complete version. I do not intend to add any more major features, so the work that remains is testing and tweaking the program before the new version is released.
https://github.com/PaulSquires/tiko/tree/development
You just need to "Download ZIP" and extract to your hard drive preferably to a location without spaces in the folder names.
DO NOT INSTALL THIS OVER YOUR EXISTING TIKO INSTALLATION
NOTE: It is a large download because it contains the full Help Centre html files, plus Tiko help html files, plus FBC toolchain. The actual Tiko Editor and DLL's are pretty small in comparison.
It is best if you unzip the program into a folder path that does not contain any spaces.
Please post here any major issues that you find so that I can polish things up before Tiko goes out for general release.
Thanks
I know that some of you try the latest Tiko versions via the github "development" branch. I can inform you that the branch now contains the latest most feature complete version. I do not intend to add any more major features, so the work that remains is testing and tweaking the program before the new version is released.
https://github.com/PaulSquires/tiko/tree/development
You just need to "Download ZIP" and extract to your hard drive preferably to a location without spaces in the folder names.
DO NOT INSTALL THIS OVER YOUR EXISTING TIKO INSTALLATION
NOTE: It is a large download because it contains the full Help Centre html files, plus Tiko help html files, plus FBC toolchain. The actual Tiko Editor and DLL's are pretty small in comparison.
It is best if you unzip the program into a folder path that does not contain any spaces.
Please post here any major issues that you find so that I can polish things up before Tiko goes out for general release.
Thanks
#40
PlanetSquires Software / Re: I sincerely hope the new v...
Last post by fbfans - August 03, 2026, 12:44:40 AMHi Paul, José Roca,
What José Roca said is exactly what I meant. The issue about UTF-16 BOM is something I mentioned in another post.
What I would like Paul to address is the problem with ANSI encoding where double-byte characters cannot be deleted properly — pressing Backspace only deletes half a character (one byte).
After carefully reading José Roca's analysis, I think it makes a lot of sense. I looked through Tiko's source code and found this:
My thought is: could we use the Windows API to get the correct ANSI code page for English, Chinese, Japanese, Korean, Big5, Shift-JIS, etc., and then set it accordingly? For example:
In short, I hope that Tiko, as a modern editor, can perfectly handle double-byte character deletion.
Thank you both for your time and effort!
What José Roca said is exactly what I meant. The issue about UTF-16 BOM is something I mentioned in another post.
What I would like Paul to address is the problem with ANSI encoding where double-byte characters cannot be deleted properly — pressing Backspace only deletes half a character (one byte).
After carefully reading José Roca's analysis, I think it makes a lot of sense. I looked through Tiko's source code and found this:
Code Select
'' IDENTIFY CHARACTERS to BE USED IN WORDS
SciMsg( pSci(i), SCI_SETWORDCHARS, 0, cast(LPARAM, @"~_\abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789") )
'' UNICODE (UTF-8 encoding)
if this.FileEncoding = FILE_ENCODING_ANSI then
SciMsg( pSci(i), SCI_SETCODEPAGE, 0, 0 )Here you are directly setting it to 0, 0. According to the Scintilla documentation, SCI_SETCODEPAGE(0) forces the editor into single-byte mode, which I believe is the root cause of the issue where double-byte characters are not correctly recognized and Backspace only deletes half a character.My thought is: could we use the Windows API to get the correct ANSI code page for English, Chinese, Japanese, Korean, Big5, Shift-JIS, etc., and then set it accordingly? For example:
Code Select
' Get current Windows system ANSI codepage
' returns 936, 950, 932, etc.
Dim As Long ansiCodePage = GetACP()
'' ANSI encoding
if this.FileEncoding = FILE_ENCODING_ANSI then
SciMsg( pSci(i), SCI_SETCODEPAGE, ansiCodePage, 0 )Would this simply solve the problem?In short, I hope that Tiko, as a modern editor, can perfectly handle double-byte character deletion.
Thank you both for your time and effort!