The upcoming release is already remarkably robust—it's a powerful IDE that fully embodies all the modern features you'd expect. With its elegant UI, full Unicode support, and seamless integration with the latest afx framework for Windows programming, Tiko has clearly positioned itself as a forward-looking tool in the FreeBASIC ecosystem.
The reason I urgently need this fix is that, under FreeBASIC, the console does not handle Chinese characters perfectly when using UTF-8 BOM or UTF-16 BOM encoding—characters often overlap or get cut off. I posted about this on the official forum but received no response. Currently, the only encoding that allows the console to display Chinese correctly is ANSI. As a result, most of my programming work is done in ANSI format.
That's precisely why the current behavior—where a single backspace deletes only half of a double-byte character under ANSI—affects me so significantly. This is the kind of problem that belonged to IDEs from 20 or 30 years ago, not to a modern editor like Tiko.
For comparison, I've tested other FreeBASIC IDEs such as FBIDE, VFBE, and VFB, and they all handle double-byte characters correctly across encoding formats. As one of the few FreeBASIC IDEs that combines high-DPI support with a clean, user-friendly experience, Tiko really should offer flawless double-byte character handling in every encoding mode.
I sincerely hope Paul will address this in a future update. I'm really looking forward to it!
Hi, this has all been fixed a couple of weeks ago.
The encoding was fixed on July 26 https://www.planetsquires.com/protect/forum/index.php?topic=4864.msg36748#msg36748
I just ran two tests on files using UTF-8 BOM and UTF-16 BOM. Both print to the console perfectly. I have attached screenshots.
These changes will be in the next Tiko update.
Hi Paul,
The easiest way to fix the ANSI double‑byte character issue in Tiko is to automatically enable Scintilla's DBCS mode whenever an ANSI file contains bytes in the DBCS range.
This requires no UI changes and works for Chinese, Japanese, Korean, Big5, Shift‑JIS, and other double‑byte encodings.
1. Detect whether the ANSI file contains double‑byte characters
When loading an ANSI file, scan the raw bytes.
If any byte is in this range:
0x81–0xFE
then the file is almost certainly encoded in a DBCS code page (GBK, Big5, Shift‑JIS, EUC‑KR, etc.). Western ANSI encodings never use bytes above 0x7F, so this detection is reliable.
2. If DBCS bytes are found, enable Scintilla's DBCS mode
Set a DBCS‑compatible code page (CP936 is the safest default)
SCI_SETCODEPAGE, 936 // GBK / CP936
CP936 is a good default because:
it supports a wide range of double‑byte characters,
it does not break Japanese or Korean text,
and it immediately fixes the backspace problem.
(You can refine this later if you want, but CP936 is enough to solve the current issue.)
Set an appropriate character set for rendering
SCI_STYLESETCHARACTERSET, STYLE_DEFAULT, SC_CHARSET_GB2312
This ensures correct width and font selection for double‑byte characters.
Please note that the issue that I'm reporting is about ANSI files that contain double‑byte characters (Chinese, Japanese, Korean, Big5, Shift‑JIS, etc.). These encodings do not use Unicode at all. They rely on DBCS (Double‑Byte Character Sets), where each character occupies two bytes.
In ANSI DBCS files, Scintilla must be explicitly switched into DBCS mode. If not, Scintilla treats each byte as a separate character, which causes the Backspace key to delete only half of a character. This is unrelated to UTF‑8/UTF‑16 and remains an issue even if Unicode printing works perfectly.
I have removed the reference to SCI_SETDBCS because it no longer exists. Scintilla automatically enables DBCS handling when the document's codepage is a DBCS codepage.
Regarding code page 936, see:
https://en.wikipedia.org/wiki/Code_page_936_(Microsoft_Windows)
Hi Paul, José Roca,
What José Roca said is exactly what I meant. The issue about UTF-16 BOM is something I mentioned in another post.
What I would like Paul to address is the problem with ANSI encoding where double-byte characters cannot be deleted properly — pressing Backspace only deletes half a character (one byte).
After carefully reading José Roca's analysis, I think it makes a lot of sense. I looked through Tiko's source code and found this:
'' IDENTIFY CHARACTERS to BE USED IN WORDS
SciMsg( pSci(i), SCI_SETWORDCHARS, 0, cast(LPARAM, @"~_\abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789") )
'' UNICODE (UTF-8 encoding)
if this.FileEncoding = FILE_ENCODING_ANSI then
SciMsg( pSci(i), SCI_SETCODEPAGE, 0, 0 )Here you are directly setting it to 0, 0. According to the Scintilla documentation, SCI_SETCODEPAGE(0) forces the editor into single-byte mode, which I believe is the root cause of the issue where double-byte characters are not correctly recognized and Backspace only deletes half a character.
My thought is: could we use the Windows API to get the correct ANSI code page for English, Chinese, Japanese, Korean, Big5, Shift-JIS, etc., and then set it accordingly? For example:
' Get current Windows system ANSI codepage
' returns 936, 950, 932, etc.
Dim As Long ansiCodePage = GetACP()
'' ANSI encoding
if this.FileEncoding = FILE_ENCODING_ANSI then
SciMsg( pSci(i), SCI_SETCODEPAGE, ansiCodePage, 0 )Would this simply solve the problem?
In short, I hope that Tiko, as a modern editor, can perfectly handle double-byte character deletion.
Thank you both for your time and effort!
Hi Paul,
I am a beginner, so please bear with me if anything I say is unclear.
I tested a modification and found that changing the wParam parameter of SCI_SETCODEPAGE from 0 to something else does work — it correctly triggers DBCS support.
Here is what I did:
In tiko.bas, I added:
Dim Shared g_SysCP As Long
g_SysCP = GetACP()Then I modified three modules:
modfindproject.inc
clsdocument.inc
modencoding.inc
In each of those modules, I replaced the 0 in SciExec( hEdit, SCI_SETCODEPAGE, 0, 0 ) with g_SysCP.
After this change, the half-character deletion issue was resolved.
I'm sharing this for your reference. It would be great if you could test this and possibly include it in the next release.
Thank you!