Main Menu

Recent posts

#21
PlanetSquires Software / Re: Some questions about usin...
Last post by José Roca - February 17, 2026, 07:43:06 PM
A little additional clarification.

If the utf-8 string is directly assigned to the DWSTRING's constructor with the code page CP_UTF8, it calls the method IsUtf8:

' ========================================================================================
' Detect if the codification is valid utf-8
' ========================================================================================
FUNCTION DWSTRING.IsUtf8 (BYREF s AS STRING) AS BOOLEAN
   IF LEN(s) = 0 THEN RETURN FALSE
   ' // Detect if it has a UTF-8 BOM
   IF LEN(s) >= 3 THEN
      IF ASC(s,1) = &hEF AND ASC(s,2) = &hBB AND ASC(s,3) = &hBF THEN RETURN TRUE
   END IF
   DIM wLen AS LONG = MultiByteToWideChar(CP_UTF8, MB_ERR_INVALID_CHARS, STRPTR(s), LEN(s), NULL, 0)
   RETURN (wLen <> 0)
END FUNCTION
' ========================================================================================

This method first checks if it has an utf-8 BOM; if not, it calls MultiByteToWideChar with the MB_ERR_INVALID_CHARS. This function fails both if it is not utf-8 and also if is malformed utf-8.

Assigning the string using the uft8 property of DWSTRING (dws.utf8 = string) assumes that you're passing a valid utff-8 string and it does not check the validity and calls WideCharToMultiByte without the MB_ERR_INVALID_CHARS flag. In Windows XP an earlier, the bad character(s) were dropped; in Vista and later, the bad character(s) are converted to U+FFFD (�).

For more information, see Raymond Chen's post "How does the MultiByteToWideChar function treat invalid characters?" at https://devblogs.microsoft.com/oldnewthing/20120504-00/?p=7703
#22
PlanetSquires Software / Re: Some questions about usin...
Last post by José Roca - February 16, 2026, 11:21:37 AM
Quote from: Paul Squires on February 15, 2026, 05:10:45 PMThanks José for the corrections. I am adding your changes to the code base. Appreciate it.

With the latest version of DWSTRING you can do:

Utf8ToAnsi

DIM dws AS DWSTRING = DWSTRING(utf8Str, CP_UTF8)
DIM ansiStr AS STRING = dws

UnicodeToUtf8

DIM strUtf8 AS STRING = dws.utf8

AnsiToUtf8

DIM dws AS DWSTRING = strAnsi
DIM strUtf8 AS STRING = dws.utf8

I also have added detection of utf8, so if you pass an ansi string to DIM dws AS DWSTRING = DWSTRING(utf8Str, CP_UTF8) it will be treated as ansi by changing CP_UTF8 to CP_ACP.

If you don't want/need a check, for speed reasons, then use:

DIM dws AS DWSTRING
dws.utf8 = strUtf8

#23
PlanetSquires Software / Re: Some questions about usin...
Last post by José Roca - February 16, 2026, 10:59:15 AM
The use of both END and SYSTEM are strongly dircouraged.

Used to exit the program, and return to the operating system. An optional integer return value can be specified to indicate an error code to the system. If no return value is given, a value of 0 is automatically returned at the end of the program.

Usage of this statement does not cleanly close scope. Local variables will not have their destructors called automatically, because FreeBASIC does not do stack unwinding. Only the destructors of global variables will be called in this case.

For this reason, it is discouraged to use End simply to mark the end of a program; the program will come to an end automatically, and in a cleaner fashion, when the last line of module-level code has executed.
#24
PlanetSquires Software / Re: Some questions about usin...
Last post by Paul Squires - February 15, 2026, 09:06:50 PM
Quote from: fbfans on February 14, 2026, 02:04:36 PM1、Uncertain issue: The "Find in Files" shortcut Ctrl+Shift+F doesn't work when the Chinese input method is active, but it works fine after switching to English mode. Ctrl+Shift is used to switch between different Chinese input methods in Chinese Windows. However, I tested other functions that also use Ctrl+Shift combinations (such as Clear All Bookmarks, Uncomment Code Block), and they all work normally. I'm just reporting this.
You can map that shortcut to another key combination if you wish.
Just use File / Preferences / Keyboard Shortcuts  and Modify the Ctrl+Shift+F menu item.

Quote<various encoding issues>
Let's discuss these issues again in the future once I release the next Tiko update. Maybe most will be fixed by the changes I have made.

Quote3、I noticed that the tooltip text in the status bar keeps refreshing when hovering the mouse.
- Fixed: Added code to prevent main statusbar tooltip text from continuously refreshing.
Thanks

Quote4、One suggestion: Regarding drag-and-drop copy, in Tiko you have to drag the selected text first and then hold Ctrl to copy, otherwise it won't work. In programs like Word, Notepad++, and EditPlus, you can either do it the Tiko way OR select text and hold Ctrl while dragging to the target location. Of course, this is just my habit—please ignore it if it's troublesome to implement.
Okay, I understand what you are trying to do, although I have not (yet) been able to implement it into Tiko. You say that it works in Notepad++? I could not get it to work there either. I tried using version 8.8.1 to so a Ctrl copy drag but it did not work. I'll keep trying to find a way to do this.

QuoteI just want Tiko to get better and better, with more complete Unicode support. Please don't mind me being so detailed. Thanks again.
Awesome, keep the suggestions coming.

QuoteLastly, here is the Chinese translation file. I've tried my best to make the translations accurate, but my English is limited.
I have added it to the Tiko distribution and will be in the next release.

QuoteNote: hz-utf8-rom.bas, sub printhz cannot be displayed in the function list.
This problem is related to you using the "End" statement in your code. It screws up the internal parser. This problem has been reported before by another user but has not yet been fixed.
You probably should not be abruptly ending your program using "End".

#25
PlanetSquires Software / Re: Some questions about usin...
Last post by Paul Squires - February 15, 2026, 05:10:45 PM
Thanks José for the corrections. I am adding your changes to the code base. Appreciate it.
#26
PlanetSquires Software / Re: Some questions about usin...
Last post by José Roca - February 15, 2026, 01:35:11 PM
I'm not planning to generate HTML, PDF or CHM documentation.
The whole point of using Markdown is precisely to avoid that kind of workflow. Markdown lets me update or correct the documentation instantly, without having to rebuild anything or maintain multiple output formats.

Anyone who wants HTML or PDF versions is, of course, free to generate them on their own using any of the many existing tools. But I won't be maintaining or producing those formats myself.

Best regards,
José
#27
PlanetSquires Software / Re: Some questions about usin...
Last post by José Roca - February 15, 2026, 01:09:32 PM
isUTF8encoded

function isUTF8encoded( byref s as string ) as boolean
    if len(s) = 0 then return false

    ' Optional: Detect UTF-8 BOM
    if len(s) >= 3 then
        if asc(s,1)=&hEF and asc(s,2)=&hBB and asc(s,3)=&hBF then return true
    end if

    dim as long wlen = MultiByteToWideChar( _
        CP_UTF8, _
        MB_ERR_INVALID_CHARS, _   ' strict validation
        strptr(s), _
        len(s), _
        NULL, _
        0)

    return (wlen <> 0)
end function
#28
PlanetSquires Software / Re: Some questions about usin...
Last post by José Roca - February 15, 2026, 12:41:04 PM
Utf8ToAnsi

function Utf8ToAnsi( byref sUtf8 as string ) as string
    dim as long wlen, alen
    dim as string sUnicode, sAnsi

    if len(sUtf8) = 0 then return ""

    '-----------------------------------------
    ' Step 1: UTF‑8 → UTF‑16 (size query)
    '-----------------------------------------
    wlen = MultiByteToWideChar( _
                CP_UTF8, _
                0, _
                strptr(sUtf8), _
                len(sUtf8), _
                NULL, _
                0)

    if wlen = 0 then return ""

    sUnicode = string(wlen * 2, 0)

    MultiByteToWideChar( _
        CP_UTF8, _
        0, _
        strptr(sUtf8), _
        len(sUtf8), _
        cast(LPWSTR, strptr(sUnicode)), _
        wlen)

    '-----------------------------------------
    ' Step 2: UTF‑16 → ANSI (size query)
    '-----------------------------------------
    alen = WideCharToMultiByte( _
                CP_ACP, _
                0, _
                cast(LPCWSTR, strptr(sUnicode)), _
                wlen, _
                NULL, _
                0, _
                NULL, _
                NULL)

    if alen = 0 then return ""

    sAnsi = string(alen, 0)

    WideCharToMultiByte( _
        CP_ACP, _
        0, _
        cast(LPCWSTR, strptr(sUnicode)), _
        wlen, _
        strptr(sAnsi), _
        alen, _
        NULL, _
        NULL)

    return sAnsi
end function
#29
PlanetSquires Software / Re: Some questions about usin...
Last post by José Roca - February 15, 2026, 12:32:02 PM
UnicodeToUtf8 has the same sizing issue

Even though LEN(wzUnicode) returns the number of UTF‑16 characters, multiplying by 2 does not guarantee enough space for UTF‑8. UTF‑8 may need 1, 2, 3 or 4 bytes per character.

function UnicodeToUtf8( byref wzUnicode as DWSTRING ) as string
    dim as long u8len
    dim as string sUtf8

    ' Step 1: get required UTF‑8 size (including null terminator)
    u8len = WideCharToMultiByte( _
                CP_UTF8, _
                0, _
                cast(LPCWSTR, *wzUnicode), _
                -1, _
                NULL, _
                0, _
                NULL, _
                NULL)

    if u8len = 0 then return ""

    ' Step 2: allocate buffer
    sUtf8 = string(u8len, 0)

    ' Step 3: convert
    WideCharToMultiByte( _
        CP_UTF8, _
        0, _
        cast(LPCWSTR, *wzUnicode), _
        -1, _
        strptr(sUtf8), _
        u8len, _
        NULL, _
        NULL)

    ' Remove null terminator
    return left(sUtf8, u8len - 1)
end function
#30
PlanetSquires Software / Re: Some questions about usin...
Last post by hajubu - February 15, 2026, 12:18:26 PM
Quote:
-------------------
I don't know what you're doing. All the files are ANSI, and the text THE SOFTWARE IS PROVIDED "AS IS" is written exactly the same way in all of them (I literally copy and paste it). Maybe you're opening the file with an editor that replaces straight quotes with typographic ones?
-------------------
In the discussion with transfering libraries to a better standard, it might a good idea to be an the safe side  for presentation the  character-depiction with a common base.

SO my first question was "why using Ansi" and not "UTF8" w.o. BOM

Nevertheless (.\AfxNova\*.* has overall app 1958 with 1759 text files '(1479*.bas, 114*.md, 166*(.bi,*.inc), 17*.xml, 12*.rc, 2*.txt)' whereof I stated are using the char(147/148) in Ascii (Ansi) .


All my Editor(s) are doing fine; therefore I could identify that case.
All depiction/presentation works in the Editors fine.

BUT not all tools showing for the char(147),char(148) inside an Ascii-text the correct left-double /right-double Quote : instead then presenting little black square ( as it above char(127) - Extended Ascii-Codes -128-255

Then the differences of  Windows-1252 using displayable characters , superset of  ISO 8859-1 / Latin-1, in terms of printable characters, but differs from the IANA's ISO-8859-1) in the range of 128-159.
Sometimes it is sufficient to select the correct code-page to be conform with wanted visualizing.

To avoid such things , I prefer using therefore the char(34) for the std. Text-Quote
to be on the safe side.

b.r. Hans

P.S. Jose : Do you have plans to adapt (convert) *.md to *.html and then to *.pdf for a common Help-System to be used outside or within Tiko?
If Yes, I can give you a hand, as I have already started with a prototype 'Windows-Procedures'( html, pdf) with TOC (table-of-content)