WinFBE 1.3.2 on GitHub (May 4, 2017)

[Paul Squires]

GitHub:  https://github.com/PaulSquires/WinFBE/releases

WinFBE Version 1.3.2

Fixed: Find dialog was not showing number of matches when word highlighted in code editor and then Find dialog displayed.
Fixed: Repaint issue with Find/Replace dialog.
Fixed: Newly added subs/functions not being added to Function List when file was saved.
Changed: Replaced all occurrences of DIR(), CURDIR(), EXEPATH() with Afx versions in order to better handle unicode languages.
Changed: Totally reworked the way WinFBE handles file encoding for ansi/utf/utf8bom/utf16bom files.

[José Roca]

Thanks for the new version.

[ganlinlao]

hi,TechSupport

When using WinFBE, a problem has been severely tortured me. As long as the code file contains any one of the Chinese characters. When I finish writing a line of code, I can't enter the ENTER key,  must use the Ctrl+enter key to end a line of code. If I enter the ENTER key to end a line of code, WinFBE immediately stops running and the WinFBE program crashes. There is no chance to save the written code. I used editors to use Enter key to end a line of code, and I am also accustomed to use the ENTER to end a line of code, when using WinFBE, I forgot to enter Control+enter, the disaster will happen immediately, the code that has been written can not be saved.
I do not know what causes, is WinFBE problem? Or is it a sciLexer.dll problem? Or the question of the Chinese input method software? But when I use notePad++, I don't have this problem, so I'm troubled.

thanks
ganlinlao

[Paul Squires]

Hi ganlinlao,

I do not know what would cause this problem. I will look at the code to see if anything special happens in WinFBE when the ENTER key is pressed and if that action is causing the problem. It is strange that Ctrl+Enter works but Enter does not. Let me investigate and I will post a response soon.

[Paul Squires]

hi,TechSupport

When using WinFBE, a problem has been severely tortured me. As long as the code file contains any one of the Chinese characters. When I finish writing a line of code, I can't enter the ENTER key,  must use the Ctrl+enter key to end a line of code. If I enter the ENTER key to end a line of code, WinFBE immediately stops running and the WinFBE program crashes. There is no chance to save the written code. I used editors to use Enter key to end a line of code, and I am also accustomed to use the ENTER to end a line of code, when using WinFBE, I forgot to enter Control+enter, the disaster will happen immediately, the code that has been written can not be saved.
I do not know what causes, is WinFBE problem? Or is it a sciLexer.dll problem? Or the question of the Chinese input method software? But when I use notePad++, I don't have this problem, so I'm troubled.

thanks
ganlinlao

When the ENTER key is pressed, WinFBE will attempt "Auto indentation" (if that option is enabled). I believe that the crash may be occurring within that code for your specific Chinese character file.

As a test, turn off "Auto indentation" in Environment Options / Code Editor, and see if the problem disappears. If it does, then we know that the problem is in that portion of code and it will make finding the specific root cause a bit easier.

[Paul Squires]

Also, could you please test the loading and saving of UTF-16 (BOM) files?

I downloaded two Chinese files from the internet (UTF8 and UTF16) in order to test. The UTF8 seems to work okay but the UTF16 does not in my test (only parts of the file loads or saves). Makes me wonder if I have problems with the clsDocument.SaveFile and/or the Utf8ToUnicode and/or UnicodeToUtf8 functions.

[ganlinlao]

hi,TechSupport

You are right, when I turn off the auto indent option, press ENTER to end a line of code, and the problem of WinFBE crashes disappears. If I turn on the Auto indent option, the problem  appear again.

Another problem, WinFBE, in ANSI mode, as long as the code contains Chinese characters, in the middle of two Chinese characters, I press the Space key, WinFBE program will collapse. However, if in Utf8Bom mode or UTF16 mode, the code contains the Chinese characters, pressing the space key, the program is not problematic.

Also, could you please test the loading and saving of UTF-16 (BOM) files?

I downloaded two Chinese files from the internet (UTF8 and UTF16) in order to test. The UTF8 seems to work okay but the UTF16 does not in my test (only parts of the file loads or saves). Makes me wonder if I have problems with the clsDocument.SaveFile and/or the Utf8ToUnicode and/or UnicodeToUtf8 functions.

1)  In UTF8Bom mode, WinFBE cannot correctly convert Chinese characters into ANSI mode, which will display garbled characters, Utf8bom and Utf16bom switch to each other, can display the Chinese character correctly.

2)  In Utf8bom mode, the file is saved as a Utf16bom format, the contents of the file can be completely preserved, with WinFBE reopening the Utf16bom format of the file, but not fully displaying the content. Part of the content was truncated and not displayed.

thanks
ganlinlao

[Paul Squires]

Seems like I was not making the UTF8 string variable large enough to handle the WideCharToMultiByte conversion.

This revised function sets the sUtf string variable to Unicode file length in bytes rather than in characters as in the previous version. This appears to fix the problem as far as I can tell.

Code Select Expand


' ========================================================================================
' Maps Unicode character string to a UTF-8 string.
' ========================================================================================
FUNCTION UnicodeToUtf8(byval pswzUnicode as wstring ptr) AS STRING
dim sUtf8 AS STRING

'Maps Unicode character string to a UTF-8 string.
sUtf8 = string(LEN(*pswzUnicode) * 2, 0)
WideCharToMultiByte(CP_UTF8, _                 'Set to UTF-8
                     0, _                       'Conversion type
                     cast(LPCWSTR, pswzUnicode), _  'Unicode string to convert
                     LEN(*pswzUnicode), _       'Length of Unicode string
                     cast(LPSTR, STRPTR(sUtf8)), _     'UTF-8 string
                     LEN(sUtf8), _              'Length of UTF-8 buffer
                     BYVAL 0, _                 'Invalid character replacement
                     BYVAL 0)                   'Replacement was used flag
FUNCTION = sUtf8

END FUNCTION



[Paul Squires]

hi,TechSupport

You can simply call me Paul 

You are right, when I turn off the auto indent option, press ENTER to end a line of code, and the problem of WinFBE crashes disappears. If I turn on the Auto indent option, the problem  appear again.

Great, thanks! I will have to find out where in that code the GPF occurs because of different character sets.

Another problem, WinFBE, in ANSI mode, as long as the code contains Chinese characters, in the middle of two Chinese characters, I press the Space key, WinFBE program will collapse. However, if in Utf8Bom mode or UTF16 mode, the code contains the Chinese characters, pressing the space key, the program is not problematic.

Should Chinese characters and editing even be done in ANSI mode? Seems like it should not. Sure, something will display on the screen but editing and saving would be problematic.

1)  In UTF8Bom mode, WinFBE cannot correctly convert Chinese characters into ANSI mode, which will display garbled characters, Utf8bom and Utf16bom switch to each other, can display the Chinese character correctly.

As above, I would not expect Chinese characters to display correctly in ANSI mode. Correct?

2)  In Utf8bom mode, the file is saved as a Utf16bom format, the contents of the file can be completely preserved, with WinFBE reopening the Utf16bom format of the file, but not fully displaying the content. Part of the content was truncated and not displayed.

Thanks! I will post an update to WinFBE with the updated UnicodeToUtf8 conversion function (that appears to fix the problem).

[Paul Squires]

New EXE's uploaded to GitHub

[ganlinlao]

thanks,Paul

In Chinese Simplified edition, the size of one character:
ANSI:    each english character size = 1 bytes (8bit), each Chinese character size = 2 bytes (16bit)
Unicode:    an English character size = 2 bytes (16bit), an chinese character size = 2 bytes (16bit)
UTF8:    each english character size = 1 bytes (8bit), each Chinese character size = 3 bytes (24bit), (21,000 chinese characters)

             A large number of non-frequently used Chinese characters are 4 bytes (57,000 chinese characters), we can ignore.

Why in ANSI mode, WinFBE can display Chinese characters, but when editing is displayed garbled? That is because a Chinese character size is 2 bytes, when deleting, inserting, appending, the cursor position must be moved +2 or-2, while deleting, inserting, appending English characters, cursor position moves +1 or-1. So you should use Len (SText) as the unit of the cursor, instead of simply using +1 or-1。
In the current winFBE version, in ansi mode, I delete a Chinese character,  must press the delete key (or backspace key) twice in the same position, winFBE will be able to correctly display Chinese characters, which is very troublesome thing.

Recheck ansiToutf8() and utf8Toansi()  function

thanks
ganlinlao

[Paul Squires]

Recheck ansiToutf8() and utf8Toansi()  function

thanks
ganlinlao

Thanks ganlinlao, I think the solution is that the code is not reloading the edit buffer with converted code whenever the file encoding changes from ANSI, UTF8, UTF16. All it is doing is setting the Scintilla settings. I need to save the current edit position, convert the text buffer with ansiToutf8() or utf8Toansi() and then finally reload the buffer with the new converted text. I will work on that today.

[Paul Squires]

I have modified the WinFBE to reload the text edit buffer whenever the file encoding changes with the newly converted text. I imagine that converting from UTF8 or UTF16 to ANSI will cause loss of fidelity in the code with unicode characters trying to be converted to ANSI.

New EXE's uploaded to GitHub.