Dear Paul,
I'm a FreeBasic hobbyist from China, and I really like Tiko—its clean design and Chinese language support are great. I've been using it for a while and have a few questions:
Some functions and procedures aren't recognized. Are there specific things I should pay attention to?
The text encoding switch in the bottom-right corner – when I click it, characters get corrupted, both Chinese and English turn into gibberish, and the source code becomes unusable. Is this indicator only meant to show the file encoding, not to change it?
Under ANSI encoding, can the editor be improved to delete one entire Chinese character at a time?
Chinese characters are double‑byte in ANSI mode, and currently pressing delete removes only half, leaving the other half as garbage.
The TODO list in the Output window shows garbled Chinese in Unicode mode. It only displays correctly in ANSI mode.
I'm also working on localizing Tiko. What does "Search Results" in the Output window refer to? I'm not sure how to use it.
Apologies if anything I've said isn't clear—I'm still learning. Thank you for your time and for this wonderful IDE.
Quote from: fbfans on February 13, 2026, 05:54:13 AMI'm a FreeBasic hobbyist from China, and I really like Tiko—its clean design and Chinese language support are great.
Awesome, happy to hear that the editor is working okay for you! I have not dealt with many Chinese programmers and sadly I do not understand the language so it's hard for me to give 100% accurate advice.
QuoteSome functions and procedures aren't recognized. Are there specific things I should pay attention to?
Most likely this is the limitation of Tiko's built in parser. There are probably edge cases that Tiko is not parsing correctly to identify them as functions. If you wish, you can post an example of the Function / End Function signature that you see as not being recognized and I'll take a look to identify the underlying problem.
QuoteThe text encoding switch in the bottom-right corner – when I click it, characters get corrupted, both Chinese and English turn into gibberish, and the source code becomes unusable. Is this indicator only meant to show the file encoding, not to change it?
It *should* try to change it.
QuoteUnder ANSI encoding, can the editor be improved to delete one entire Chinese character at a time?
Chinese characters are double‑byte in ANSI mode, and currently pressing delete removes only half, leaving the other half as garbage.
I don't think that you should use ANSI mode with Chinese characters. You are better off using UTF-8 or UTF-16 (unicode) because ANSI is only designed for single character languages like English and other Latin based languages.
QuoteThe TODO list in the Output window shows garbled Chinese in Unicode mode. It only displays correctly in ANSI mode.
Ahhh... I just checked Tiko's internal database and it looks like years ago that I must have used fixed ZSTRING elements instead of WSTRING !!! That could definitely be the reason why the TODO text is wrong. It could also be why the functions/procedures are not being displayed. I will change this and post an update.
QuoteI'm also working on localizing Tiko. What does "Search Results" in the Output window refer to? I'm not sure how to use it.
Search Results are those found when you perform a "Find in Files" found under the Edit menu (or Ctrl+Shift+F)
I have made changes to Tiko's internal parser and database to use AfxNova's dynamic wstring DWSTRING variable type. This *should* help with storing the Chinese text for TODO, etc.
The changes (and tiko.exe) can be found on the Development branch. That branch will eventually be merged to the Master branch before the next Tiko update is released.
https://github.com/PaulSquires/tiko/tree/development
Thanks, Paul. You've been really helpful. I truly appreciate your patience and detailed replies to a beginner like me—I feel really honored and flattered.I'm not familiar with GitHub, so I'll just wait for your next version update. Right now, there are only minor issues that affect the experience a little but don't prevent usage, and I can work around them.I'm really looking forward to the powerful code completion in the new version. I find it difficult to type AFXNova-related functions because they are too long and hard to remember (my English isn't very good).
I'll keep reporting small issues with Tiko, and I hope I don't bother you:
1、Uncertain issue: The "Find in Files" shortcut Ctrl+Shift+F doesn't work when the Chinese input method is active, but it works fine after switching to English mode. Ctrl+Shift is used to switch between different Chinese input methods in Chinese Windows. However, I tested other functions that also use Ctrl+Shift combinations (such as Clear All Bookmarks, Uncomment Code Block), and they all work normally. I'm just reporting this.
2、The following are all encoding issues with double-byte characters like Chinese. You might not have the environment to test them, so I took screenshots. Although they don't affect Latin-language users at all, I still hope you can add them to your to-do list and fix them when you have time.
(1) Find in Files cannot search for Chinese characters in Unicode-encoded files; it only works in ANSI format. As you can see from the filenames in my screenshot, several files starting with "hz" have identical content but different encodings (maybe your recent fixes have addressed this using DWSTRING, but I haven't been able to test it).
(2) When using split view, garbled characters appear in ANSI-encoded files. It seems that in horizontal split, the left pane shows garbled text; in vertical split, the top pane shows garbled text. The garbled characters disappear when restoring to a single window. No such issue with Unicode files.
3、I noticed that the tooltip text in the status bar keeps refreshing when hovering the mouse. Is this the default behavior of AFXNova functions? I've been studying the Limit function recently, so I'm quite sensitive to similar issues, haha.
4、One suggestion: Regarding drag-and-drop copy, in Tiko you have to drag the selected text first and then hold Ctrl to copy, otherwise it won't work. In programs like Word, Notepad++, and EditPlus, you can either do it the Tiko way OR select text and hold Ctrl while dragging to the target location. Of course, this is just my habit—please ignore it if it's troublesome to implement.
I just want Tiko to get better and better, with more complete Unicode support. Please don't mind me being so detailed. Thanks again.
Lastly, here is the Chinese translation file. I've tried my best to make the translations accurate, but my English is limited. Please forgive any inaccuracies.
Note: hz-utf8-rom.bas, sub printhz cannot be displayed in the function list.
Welcome to the forum.
Never use ANSI with Chinese.
dim as string hz1,hz2
hz1 = "123微软专用操作系统安装工具456！"
hz2 = "abcgq这是显示汉字的子程序def"
The STRING data type is ansi. Therefore, the Chinese characters will be comverted to ?? or similar. Use WSTRING or AfxNova's DWSTRING.
The use of SYSTEM and END are discouraged. Please, read the manual.
Hi Jose,
reading just the discussion here, I have stated some deviations in the 'AfxNova'-Files.
When introducing the new generic Info Header in the *.bi,*.inc you used files in Ansii-format.
Would it not better using UTF8 , or even UTF16-LE - as long "GitHub" does not interfere ,
to make the life easier for the presentation in "old" Editors.
b.r. Hans
P.S. Besides these small glitches,
I do really appreciate the effort and amount of hours , put in the work for the community-,
- especially here in the Tiko-Base .
Thanks so much ! Merci beaucoup ! Muchas gracias ! Vielen Dank !
'---------------------------------------------------------------------------
FAQ for JOSE - AfxNova Files ( except doc-files (.md)
'---------------------------------------------------------------------------
1st) Why using Ansii-Text at all ,
2nd) with [left_quote/right_quote],[chr(147).. ..chr(148)]
3rd) Would it not better, using (at least) Utf8-text throughout all AfxNova-files
IMHO::
--> advantage - UTF8 depiction/presentation would be clearer for all cases (EN-us).
BUT ....
--> typically:: UTF16 for East Asian languages ,most: one 16 bit word, rare: two 16 bit words
--> and for Western languages UTF-8 (ASCII is equivalent over the ASCII range (0-127) )
--> Processing: UTF-16 for user-mode applications is easier than processing UTF-8
--> Windows(today) :: UTF16 ,
--> (UTF32 rarely used , where codepoints ask for....)
::: Unicode .version....17...2025 AND goes on ...18.... t.b.d.
'-----------------------------------------------------------------------------
'
'*************************************
--> here my findings in AFxNova-files for the quotes around a 'tag'
'************************************
'RE(\sMIT\s) License
' in 99 files
'---
' i.e. ANSI TEXT {MIT} and "AS IS" e.g. with [Double_quotes], [chr(34).. ..chr(34)]
' Written in 2017... by José Roca -->'4a 6f 73 E9'::'José'
'' THE SOFTWARE IS PROVIDED "AS IS", -->'22 41 53 20 49 53 22'::'"AS IS"'
'---
'...\AfxNova\AfxComplex.inc
'...\AfxNova\CComplex.inc
'---------------------------
'---------------------------
'RE (\sMIT\s)|("AS IS")
'in 97 files
---
' i.e. ANSI TEXT ... "AS IS" with [left_quote/right_quote],[chr(147).. ..chr(148)]
' Copyright (c) 2025 José Roca --> '4a 6f 73 e9'::'José'
' License: Distributed under the MIT license.
' THE SOFTWARE IS PROVIDED "AS IS", -->'93 41 53 20 49 53 94 2c'::'"AS IS",'
'
'***************************
more details in the attachment (list of the ANSII-AfxNova-Files)
I have had problems with not ansi files uploaded to GitHub when someone downloads them as a zip file, specially if they are in utf-16. The problem seems to be caused by the BOM.
Quote from: Paul Squires on February 14, 2026, 09:28:13 AMI have made changes to Tiko's internal parser and database to use AfxNova's dynamic wstring DWSTRING variable type. This *should* help with storing the Chinese text for TODO, etc.
The changes (and tiko.exe) can be found on the Development branch. That branch will eventually be merged to the Master branch before the next Tiko update is released.
https://github.com/PaulSquires/tiko/tree/development
Paul, if you're using FreeBASIC's UTF‑8 conversion routines, I strongly suggest switching to MultiByteToWideChar and WideCharToMultiByte. FreeBASIC's routines are notoriously flaky and tend to corrupt characters.
Thanks José,
Yes, I am using MultiByteToWideChar and WideCharToMultiByte for the conversions. I have posted the routines below that I am using for the conversions back and forth.
' ========================================================================================
' Maps UTF-8 string to Ansi string.
' ========================================================================================
function Utf8ToAnsi(byref strUtf8 as string) as string
dim i as long ' // Loop counter
dim strAscii as string ' // Ascii string
dim idx as long ' // Position in the string
dim c as long ' // ASCII code
dim b2 as long ' // Second byte
dim fSkipChar as boolean ' // Flag
if len(strUtf8) = 0 then exit function
' // The maximum length of the translated string will be
' // the same as the length of the original string.
' // We are pre-allocating the buffer for faster operation
' // than concatenating each character one by one.
strAscii = space(len(strUtf8))
' // Intialize index position in the string buffer
' // used to store the converted Ascii string
idx = 1
' // Examine the contents of each character in the UTF-8 encoded string
for i = 1 to len(strUtf8)
' // If fSkipChar is set we have to skip this character
if fSkipChar then
fSkipChar = 0
continue for
end if
' // Get the Ascii code of the character
c = asc(mid(strUtf8, i, 1))
' // If it is betwen 0 and 127...
if c < 128 then
' // ...we simply copy it to the string buffer...
mid(strAscii, idx, 1) = mid(strUtf8, idx, 1)
' // ...and increase the position by 1.
idx = idx + 1
elseif c < 224 then
' // We need to join this byte and the next byte.
b2 = asc(mid(strUtf8, i + 1, 1))
if b2 > 127 then
c = (c - 192) * 64 + (b2 - 128)
mid(strAscii, idx, 1) = chr(c)
' // Set the flag to skip the next character
fSkipChar = true
' // Increase the position by 1.
idx = idx + 1
end if
end if
next
' // Return the string
function = left(strAscii, idx - 1)
end function
' ========================================================================================
' Maps Unicode character string to a UTF-8 string.
' ========================================================================================
function UnicodeToUtf8( byval wzUnicode as DWSTRING ) as string
dim sUtf8 as string
' Maps Unicode character string to a UTF-8 string.
sUtf8 = string(len(wzUnicode) * 2, 0)
dim as long bytesWritten = _
WideCharToMultiByte ( _
CP_UTF8, _ 'Set to UTF-8
0, _ 'Conversion type
cast(LPCWSTR, wzUnicode.vptr), _ 'Unicode string to convert
len(wzUnicode), _ 'Length of Unicode string
cast(LPSTR, strptr(sUtf8)), _ 'UTF-8 string
len(sUtf8), _ 'Length of UTF-8 buffer
byval 0, _ 'Invalid character replacement
byval 0) 'Replacement was used flag
function = left(sUtf8, bytesWritten)
end function
' ========================================================================================
' Maps Ansi character string to a UTF-8 string.
' ========================================================================================
function AnsiToUtf8( byref sAnsi as string ) as string
dim sUnicode as string
dim sUtf8 as string
'Maps Ansi character string to a UTF-8 string.
'Step one, convert to UNICODE
sUnicode = string(len(sAnsi) * 2, 0)
MultiByteToWideChar(CP_ACP, _ 'System default Windows ANSI code page
MB_PRECOMPOSED, _ 'Conversion type
cast(LPCSTR, strptr(sAnsi)), _ 'ANSI string to convert
len(sAnsi), _ 'Lenght of ANSI string
cast(LPWSTR, strptr(sUnicode)), _ 'Unicode string
len(sUnicode)) 'Lenght of Unicode buffer
'Step two, convert to UTF-8
sUtf8 = string(len(sAnsi), 0)
WideCharToMultiByte(CP_UTF8, _ 'Set to UTF-8
0, _ 'Conversion type
cast(LPCWSTR, strptr(sUnicode)), _ 'Unicode string to convert
len(sUnicode) / 2, _ 'Lenght of Unicode string
cast(LPSTR, strptr(sUtf8)), _ 'UTF-8 string
len(sUtf8), _ 'Length of UTF-8 buffer
byval 0, _ 'Invalid character replacement
byval 0) 'Replacement was used flag
function = sUtf8
end function
' ========================================================================================
' Determine if a string is likely UTF-8 encoded
' ========================================================================================
function isUTF8encoded(byref text as string) as boolean
dim as ubyte ptr p = strptr(text)
dim as integer nlen = len(text)
dim as integer i = 0
dim as boolean hasUTF8Sequences = false
while i < nlen
dim as ubyte c = p[i]
if c < &h80 then
' ASCII character
i += 1
elseif (c and &hE0) = &hC0 then
' 2-byte UTF-8 sequence
if i + 1 >= nlen then return false
if (p[i + 1] and &hC0) <> &h80 then return false
if c < &hC2 then return false
hasUTF8Sequences = true
i += 2
elseif (c and &hF0) = &hE0 then
' 3-byte UTF-8 sequence
if i + 2 >= nlen then return false
if (p[i + 1] and &hC0) <> &h80 then return false
if (p[i + 2] and &hC0) <> &h80 then return false
hasUTF8Sequences = true
i += 3
elseif (c and &hF8) = &hF0 then
' 4-byte UTF-8 sequence
if i + 3 >= nlen then return false
if (p[i + 1] and &hC0) <> &h80 then return false
if (p[i + 2] and &hC0) <> &h80 then return false
if (p[i + 3] and &hC0) <> &h80 then return false
hasUTF8Sequences = true
i += 4
else
' Invalid byte for UTF-8
return false
end if
wend
return hasUTF8Sequences
end function
Quote'*************************************
--> here my findings in AFxNova-files for the quotes around a 'tag'
'************************************
'RE(\sMIT\s) License
' in 99 files
'---
' i.e. ANSI TEXT {MIT} and "AS IS" e.g. with [Double_quotes], [chr(34).. ..chr(34)]
' Written in 2017... by José Roca -->'4a 6f 73 E9'::'José'
'' THE SOFTWARE IS PROVIDED "AS IS", -->'22 41 53 20 49 53 22'::'"AS IS"'
'---
'...\AfxNova\AfxComplex.inc
'...\AfxNova\CComplex.inc
'---------------------------
'---------------------------
'RE (\sMIT\s)|("AS IS")
'in 97 files
---
' i.e. ANSI TEXT ... "AS IS" with [left_quote/right_quote],[chr(147).. ..chr(148)]
' Copyright (c) 2025 José Roca --> '4a 6f 73 e9'::'José'
' License: Distributed under the MIT license.
' THE SOFTWARE IS PROVIDED "AS IS", -->'93 41 53 20 49 53 94 2c'::'"AS IS",'
'
I don't know what you're doing. All the files are ANSI, and the text THE SOFTWARE IS PROVIDED "AS IS" is written exactly the same way in all of them (I literally copy and paste it). Maybe you're opening the file with an editor that replaces straight quotes with typographic ones?
AnsiToUtf8
1. You allocate the UTF‑8 buffer as `len(sAnsi)`, but UTF‑8 may require up to 3 bytes per ANSI character. This means WideCharToMultiByte can truncate or produce invalid output.
2. You never call WideCharToMultiByte with a NULL buffer to obtain the correct output size. This is the recommended and safe way to use the API.
3. You don't check the return values of MultiByteToWideChar or WideCharToMultiByte, so conversion errors go unnoticed.
My code first calls MultiByteToWideChar/WideCharToMultiByte with a NULL buffer to get the exact size needed, allocates the correct amount of memory, and only then performs the conversion. That's why it doesn't corrupt characters.
function AnsiToUtf8( byref sAnsi as string ) as string
dim as integer wlen, u8len
dim as string sUnicode, sUtf8
if len(sAnsi) = 0 then return ""
' Step 1: ANSI ? UTF-16 (size query)
wlen = MultiByteToWideChar(CP_ACP, 0, strptr(sAnsi), len(sAnsi), NULL, 0)
if wlen = 0 then return ""
sUnicode = string(wlen * 2, 0)
if MultiByteToWideChar(CP_ACP, 0, strptr(sAnsi), len(sAnsi), cast(LPWSTR, strptr(sUnicode)), wlen) = 0 then
return ""
end if
' Step 2: UTF-16 ? UTF-8 (size query)
u8len = WideCharToMultiByte(CP_UTF8, 0, cast(LPCWSTR, strptr(sUnicode)), wlen, NULL, 0, NULL, NULL)
if u8len = 0 then return ""
sUtf8 = string(u8len, 0)
if WideCharToMultiByte(CP_UTF8, 0, cast(LPCWSTR, strptr(sUnicode)), wlen, strptr(sUtf8), u8len, NULL, NULL) = 0 then
return ""
end if
return sUtf8
end function
Quote:
-------------------
I don't know what you're doing. All the files are ANSI, and the text THE SOFTWARE IS PROVIDED "AS IS" is written exactly the same way in all of them (I literally copy and paste it). Maybe you're opening the file with an editor that replaces straight quotes with typographic ones?
-------------------
In the discussion with transfering libraries to a better standard, it might a good idea to be an the safe side for presentation the character-depiction with a common base.
SO my first question was "why using Ansi" and not "UTF8" w.o. BOM
Nevertheless (.\AfxNova\*.* has overall app 1958 with 1759 text files '(1479*.bas, 114*.md, 166*(.bi,*.inc), 17*.xml, 12*.rc, 2*.txt)' whereof I stated are using the char(147/148) in Ascii (Ansi) .
All my Editor(s) are doing fine; therefore I could identify that case.
All depiction/presentation works in the Editors fine.
BUT not all tools showing for the char(147),char(148) inside an Ascii-text the correct left-double /right-double Quote : instead then presenting little black square ( as it above char(127) - Extended Ascii-Codes -128-255
Then the differences of Windows-1252 using displayable characters , superset of ISO 8859-1 / Latin-1, in terms of printable characters, but differs from the IANA's ISO-8859-1) in the range of 128-159.
Sometimes it is sufficient to select the correct code-page to be conform with wanted visualizing.
To avoid such things , I prefer using therefore the char(34) for the std. Text-Quote
to be on the safe side.
b.r. Hans
P.S. Jose : Do you have plans to adapt (convert) *.md to *.html and then to *.pdf for a common Help-System to be used outside or within Tiko?
If Yes, I can give you a hand, as I have already started with a prototype 'Windows-Procedures'( html, pdf) with TOC (table-of-content)
UnicodeToUtf8 has the same sizing issue
Even though LEN(wzUnicode) returns the number of UTF‑16 characters, multiplying by 2 does not guarantee enough space for UTF‑8. UTF‑8 may need 1, 2, 3 or 4 bytes per character.
function UnicodeToUtf8( byref wzUnicode as DWSTRING ) as string
dim as long u8len
dim as string sUtf8
' Step 1: get required UTF‑8 size (including null terminator)
u8len = WideCharToMultiByte( _
CP_UTF8, _
0, _
cast(LPCWSTR, *wzUnicode), _
-1, _
NULL, _
0, _
NULL, _
NULL)
if u8len = 0 then return ""
' Step 2: allocate buffer
sUtf8 = string(u8len, 0)
' Step 3: convert
WideCharToMultiByte( _
CP_UTF8, _
0, _
cast(LPCWSTR, *wzUnicode), _
-1, _
strptr(sUtf8), _
u8len, _
NULL, _
NULL)
' Remove null terminator
return left(sUtf8, u8len - 1)
end function
Utf8ToAnsi
function Utf8ToAnsi( byref sUtf8 as string ) as string
dim as long wlen, alen
dim as string sUnicode, sAnsi
if len(sUtf8) = 0 then return ""
'-----------------------------------------
' Step 1: UTF‑8 → UTF‑16 (size query)
'-----------------------------------------
wlen = MultiByteToWideChar( _
CP_UTF8, _
0, _
strptr(sUtf8), _
len(sUtf8), _
NULL, _
0)
if wlen = 0 then return ""
sUnicode = string(wlen * 2, 0)
MultiByteToWideChar( _
CP_UTF8, _
0, _
strptr(sUtf8), _
len(sUtf8), _
cast(LPWSTR, strptr(sUnicode)), _
wlen)
'-----------------------------------------
' Step 2: UTF‑16 → ANSI (size query)
'-----------------------------------------
alen = WideCharToMultiByte( _
CP_ACP, _
0, _
cast(LPCWSTR, strptr(sUnicode)), _
wlen, _
NULL, _
0, _
NULL, _
NULL)
if alen = 0 then return ""
sAnsi = string(alen, 0)
WideCharToMultiByte( _
CP_ACP, _
0, _
cast(LPCWSTR, strptr(sUnicode)), _
wlen, _
strptr(sAnsi), _
alen, _
NULL, _
NULL)
return sAnsi
end function
isUTF8encoded
function isUTF8encoded( byref s as string ) as boolean
if len(s) = 0 then return false
' Optional: Detect UTF-8 BOM
if len(s) >= 3 then
if asc(s,1)=&hEF and asc(s,2)=&hBB and asc(s,3)=&hBF then return true
end if
dim as long wlen = MultiByteToWideChar( _
CP_UTF8, _
MB_ERR_INVALID_CHARS, _ ' strict validation
strptr(s), _
len(s), _
NULL, _
0)
return (wlen <> 0)
end function
I'm not planning to generate HTML, PDF or CHM documentation.
The whole point of using Markdown is precisely to avoid that kind of workflow. Markdown lets me update or correct the documentation instantly, without having to rebuild anything or maintain multiple output formats.
Anyone who wants HTML or PDF versions is, of course, free to generate them on their own using any of the many existing tools. But I won't be maintaining or producing those formats myself.
Best regards,
José
Thanks José for the corrections. I am adding your changes to the code base. Appreciate it.
Quote from: fbfans on February 14, 2026, 02:04:36 PM1、Uncertain issue: The "Find in Files" shortcut Ctrl+Shift+F doesn't work when the Chinese input method is active, but it works fine after switching to English mode. Ctrl+Shift is used to switch between different Chinese input methods in Chinese Windows. However, I tested other functions that also use Ctrl+Shift combinations (such as Clear All Bookmarks, Uncomment Code Block), and they all work normally. I'm just reporting this.
You can map that shortcut to another key combination if you wish.
Just use File / Preferences / Keyboard Shortcuts and Modify the Ctrl+Shift+F menu item.
Quote<various encoding issues>
Let's discuss these issues again in the future once I release the next Tiko update. Maybe most will be fixed by the changes I have made.
Quote3、I noticed that the tooltip text in the status bar keeps refreshing when hovering the mouse.
- Fixed: Added code to prevent main statusbar tooltip text from continuously refreshing.
Thanks
Quote4、One suggestion: Regarding drag-and-drop copy, in Tiko you have to drag the selected text first and then hold Ctrl to copy, otherwise it won't work. In programs like Word, Notepad++, and EditPlus, you can either do it the Tiko way OR select text and hold Ctrl while dragging to the target location. Of course, this is just my habit—please ignore it if it's troublesome to implement.
Okay, I understand what you are trying to do, although I have not (yet) been able to implement it into Tiko. You say that it works in Notepad++? I could not get it to work there either. I tried using version 8.8.1 to so a Ctrl copy drag but it did not work. I'll keep trying to find a way to do this.
QuoteI just want Tiko to get better and better, with more complete Unicode support. Please don't mind me being so detailed. Thanks again.
Awesome, keep the suggestions coming.
QuoteLastly, here is the Chinese translation file. I've tried my best to make the translations accurate, but my English is limited.
I have added it to the Tiko distribution and will be in the next release.
QuoteNote: hz-utf8-rom.bas, sub printhz cannot be displayed in the function list.
This problem is related to you using the "End" statement in your code. It screws up the internal parser. This problem has been reported before by another user but has not yet been fixed.
You probably should not be abruptly ending your program using "End".
The use of both END and SYSTEM are strongly dircouraged.
Used to exit the program, and return to the operating system. An optional integer return value can be specified to indicate an error code to the system. If no return value is given, a value of 0 is automatically returned at the end of the program.
Usage of this statement does not cleanly close scope. Local variables will not have their destructors called automatically, because FreeBASIC does not do stack unwinding. Only the destructors of global variables will be called in this case.
For this reason, it is discouraged to use End simply to mark the end of a program; the program will come to an end automatically, and in a cleaner fashion, when the last line of module-level code has executed.
Quote from: Paul Squires on February 15, 2026, 05:10:45 PMThanks José for the corrections. I am adding your changes to the code base. Appreciate it.
With the latest version of DWSTRING you can do:
Utf8ToAnsiDIM dws AS DWSTRING = DWSTRING(utf8Str, CP_UTF8)
DIM ansiStr AS STRING = dws
UnicodeToUtf8DIM strUtf8 AS STRING = dws.utf8
AnsiToUtf8DIM dws AS DWSTRING = strAnsi
DIM strUtf8 AS STRING = dws.utf8
I also have added detection of utf8, so if you pass an ansi string to DIM dws AS DWSTRING = DWSTRING(utf8Str, CP_UTF8) it will be treated as ansi by changing CP_UTF8 to CP_ACP.
If you don't want/need a check, for speed reasons, then use:
DIM dws AS DWSTRING
dws.utf8 = strUtf8
A little additional clarification.
If the utf-8 string is directly assigned to the DWSTRING's constructor with the code page CP_UTF8, it calls the method IsUtf8:
' ========================================================================================
' Detect if the codification is valid utf-8
' ========================================================================================
FUNCTION DWSTRING.IsUtf8 (BYREF s AS STRING) AS BOOLEAN
IF LEN(s) = 0 THEN RETURN FALSE
' // Detect if it has a UTF-8 BOM
IF LEN(s) >= 3 THEN
IF ASC(s,1) = &hEF AND ASC(s,2) = &hBB AND ASC(s,3) = &hBF THEN RETURN TRUE
END IF
DIM wLen AS LONG = MultiByteToWideChar(CP_UTF8, MB_ERR_INVALID_CHARS, STRPTR(s), LEN(s), NULL, 0)
RETURN (wLen <> 0)
END FUNCTION
' ========================================================================================
This method first checks if it has an utf-8 BOM; if not, it calls MultiByteToWideChar with the MB_ERR_INVALID_CHARS. This function fails both if it is not utf-8 and also if is malformed utf-8.
Assigning the string using the uft8 property of DWSTRING (dws.utf8 = string) assumes that you're passing a valid utff-8 string and it does not check the validity and calls WideCharToMultiByte without the MB_ERR_INVALID_CHARS flag. In Windows XP an earlier, the bad character(s) were dropped; in Vista and later, the bad character(s) are converted to U+FFFD (�).
For more information, see Raymond Chen's post "How does the MultiByteToWideChar function treat invalid characters?" at https://devblogs.microsoft.com/oldnewthing/20120504-00/?p=7703
I have asked to CoPilot about porting DWSTRING to Linux to end with the UTF-8 headaches that the users of that platform have.
And it has replied:
🌟 Why a Linux version of DWSTRING is straightforward
1. FreeBasic already gives you UTF‑32 on Linux
This is the jackpot.
WSTRING on Linux = UCS‑4 = UTF‑32.
So your internal buffer becomes:
one code point per element
no surrogate pairs
no variable‑length sequences
no special cases
It's even simpler than UTF‑16.
2. Linux already provides the conversion functions you need
Windows:
MultiByteToWideChar
WideCharToMultiByte
Linux equivalents:
mbstowcs / wcstombs
or iconv for more control
These do the same job.
3. The rest of DWSTRING is platform‑neutral
Your class logic:
constructors
operators
buffer management
UTF‑8 validation
integration with FB string functions
...all of that stays exactly the same.
4. No external dependencies means no obstacles
Because DWSTRING is:
one file
no DLLs
no libraries
no Windows‑only baggage
...it ports cleanly.
🌱 Why it feels too easy
Because most developers approach UTF‑8 the hard way:
manipulating UTF‑8 directly
rewriting MID, LEFT, RIGHT, INSTR
building ULeft, URight, UInstr, UReplace, URemove, UInsert
getting stuck on UMid
fighting variable‑length encodings
You avoided all of that by stepping outside the UTF‑8 trap entirely.
Your model:
UTF‑8 → fixed‑width Unicode → process → fixed‑width Unicode → UTF‑8
...is the same one used by:
Windows
.NET
Java
Qt
Python (on Linux)
ICU
It's easy because it's correct.
🎯 And yes — with or without AI, it's absolutely doable
An AI tool could help generate the Linux version, but the truth is:
your design is already portable
the platform‑specific parts are tiny
the logic is universal
the class is dependency‑free
So whether you do it manually or with assistance, the path is clear.
@Paul, José
Thanks.
I have received your reply. The issue with the function name not being recognized is actually quite simple—I can resolve it by using "system." I'm just not entirely sure why this became a problem in the first place.
I'm not sure why I seem to be encountering so many issues, while other users don't seem to have any problems with tiko. I hope this doesn't come across the wrong way, Paul.
1、Regarding the function list for files: Shouldn't clicking on it allow you to expand or collapse the list for each file? Next to the filename, there's an arrow, but clicking it jumps to the function list of the corresponding file instead of collapsing it. In contrast, the bookmark list follows the expected behavior—clicking expands or collapses it properly.
2、The scrollbar's drag activation area is too narrow. When dragging the scrollbar, if the mouse moves even slightly outside of it, it immediately loses focus and the scrollbar disappears from view, even though it can still be dragged.
3、Regarding bookmarks: Normally, when a bookmark is set, it should appear in the sidebar's bookmark tab. Currently, only bookmarks set via Ctrl+F2 appear automatically. Bookmarks set with the mouse do not—they only show up if I click the "..." next to the sidebar bookmarks and select "Expand All." In your version 1.3 update, you added the option to set breakpoints. If you intend for a mouse click to set a breakpoint, I suggest either disabling the option to set bookmarks with the mouse (by removing the relevant setting) or assigning mouse double-click or another key to set breakpoints to avoid conflicts. Otherwise, a logic for adding mouse-set bookmarks to the list should be implemented.
Additionally, I revisited the translation and am unsure if "clear bookmarks" in the sidebar and menu bar are meant to have the same meaning. However, the menu bar option only clears bookmarks for the current file, while the sidebar option clears bookmarks for all files. The difference is quite significant—they are not performing the same action, so the translation should reflect that distinction.
I feel that José's dwstring is very powerful. The transcoding function he demonstrated is logically clear, concise, robust, and almost like a tutorial—definitely worth learning from. I suggest Paul consider adopting this approach, as it could greatly simplify Unicode-related issues. It's highly recommendable.
Regarding Chinese character encoding conversion, Paul may not have the conditions to test it. If you need me to assist with testing, feel free to reach out—I'm more than happy to help. Email: erliwuwei@gmail.com.
Quote from: fbfans on February 20, 2026, 01:16:03 PMI'm just not entirely sure why this became a problem in the first place.
It is a well known Tiko parsing problem. The "end" interferes with the parser determining where the Sub/Function ends because the parser expects an End Sub or an End Function.
Quote1、Regarding the function list for files: Shouldn't clicking on it allow you to expand or collapse the list for each file? Next to the filename, there's an arrow, but clicking it jumps to the function list of the corresponding file instead of collapsing it. In contrast, the bookmark list follows the expected behavior—clicking expands or collapses it properly.
Yes, you're probably right about this one. I will look to see why I implemented it the current way and change it if necessary.
Quote2、The scrollbar's drag activation area is too narrow. When dragging the scrollbar, if the mouse moves even slightly outside of it, it immediately loses focus and the scrollbar disappears from view, even though it can still be dragged.
This was actually fixed (finally) yesterday. See this post: https://www.freebasic.net/forum/viewtopic.php?p=310481#p310481
Quote3、Regarding bookmarks: Normally, when a bookmark is set, it should appear in the sidebar's bookmark tab. Currently, only bookmarks set via Ctrl+F2 appear automatically. Bookmarks set with the mouse do not—they only show up if I click the "..." next to the sidebar bookmarks and select "Expand All."
Yes, there does seem to be something strange going on there.
QuoteIn your version 1.3 update, you added the option to set breakpoints. If you intend for a mouse click to set a breakpoint, I suggest either disabling the option to set bookmarks with the mouse (by removing the relevant setting) or assigning mouse double-click or another key to set breakpoints to avoid conflicts. Otherwise, a logic for adding mouse-set bookmarks to the list should be implemented.
Check out the option: Environment Options / Code Editor / "Margin click toggles Breakpoint rather than Bookmark".
QuoteAdditionally, I revisited the translation and am unsure if "clear bookmarks" in the sidebar and menu bar are meant to have the same meaning. However, the menu bar option only clears bookmarks for the current file, while the sidebar option clears bookmarks for all files. The difference is quite significant—they are not performing the same action, so the translation should reflect that distinction.
Clicking the "..." and selecting "clear bookmarks" should remove all bookmarks from ALL files. Right clicking on a filename within the list and selecting "clear bookmarks" should remove bookmarks from that file only. Maybe I should rename them "Clear Bookmarks All Files", and " Clear Bookmarks Selected File", or something similar.
Hi Paul, thanks for your reply.
QuoteCheck this option: Environment Options / Code Editor / "Click margin to toggle breakpoints instead of bookmarks".
What I mean is, the current program logic is that clicking the line number area either sets a breakpoint or a bookmark—you can only choose one. Why not set breakpoints with a left click and bookmarks with a right click, or use a double click and other mouse actions instead?
Another point is about the real-time sync and refresh issue of the sidebar. I closed some files, but their entries still remain in the sidebar. Theoretically, the closed files should no longer be displayed there.
I downloaded the tiko.exe file from the development branch directory on GitHub at https://github.com/PaulSquires/tiko/tree/development, and it prompts an incompatibility with my system. I'm using 64-bit Windows 11.
Also, I just noticed there are so many discussions about Tiko on the FreeBASIC forum. Should I post my questions there from now on?