PlanetSquires Forums

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1] 2

Author Topic: Listbox - unicode items  (Read 273 times)

Bumblebee

  • Junior Member
  • **
  • Posts: 75
Listbox - unicode items
« on: April 07, 2021, 11:40:21 AM »

I may not have noticed this before, but does the listbox support unicode characters?
Using the latest version 2.2.0

e.g. lgance is displayed as Élégance
« Last Edit: April 07, 2021, 11:42:21 AM by Bumblebee »
Logged

Paul Squires

  • Administrator
  • Guru Member
  • *****
  • Posts: 9386
  • Windows 10
    • PlanetSquires Software
Re: Listbox - unicode items
« Reply #1 on: April 07, 2021, 12:07:55 PM »

Hi, yes the Listbox code does support unicode. The following correctly displays your code:

Code: [Select]
   for i as long = 0 to 5
      frmMain.List1.Items.Add( "lgance" & i )
   next
Logged
Paul Squires
PlanetSquires Software
WinFBE Editor and Visual Designer

Bumblebee

  • Junior Member
  • **
  • Posts: 75
Re: Listbox - unicode items
« Reply #2 on: April 07, 2021, 12:19:13 PM »

Might be a problem with the string variables I'm using.
Do I need to use CWSTR to preserve unicode characters?
Logged

Paul Squires

  • Administrator
  • Guru Member
  • *****
  • Posts: 9386
  • Windows 10
    • PlanetSquires Software
Re: Listbox - unicode items
« Reply #3 on: April 07, 2021, 12:29:40 PM »

CWSTR will work. WSTRING will work as well.
I doubt that STRING will work reliably.
Logged
Paul Squires
PlanetSquires Software
WinFBE Editor and Visual Designer

Bumblebee

  • Junior Member
  • **
  • Posts: 75
Re: Listbox - unicode items
« Reply #4 on: April 07, 2021, 01:07:50 PM »

I use regular string variables to write the file that contains unicode characters. It seems to work with no issues.
Nor did I specify utf-x encoding when writing the file.

Replacing string with cwstr causes an invalid data type in the input statement.
Logged

Jos Roca

  • Guru Member
  • *****
  • Posts: 3325
Re: Listbox - unicode items
« Reply #5 on: April 07, 2021, 02:08:21 PM »

What do you understand by unicode characters? Accented characters like , , , , aren't unicode.

FB ansi variables can't hold unicode characters like Серге́й Серге́евич Проко́фьев.

Bumblebee

  • Junior Member
  • **
  • Posts: 75
Re: Listbox - unicode items
« Reply #6 on: April 07, 2021, 02:15:20 PM »

I'm having an issue reading accented characters, as per the example.
According to Notepad, the file I'm parsing is in UTF-8 Unix (LF)

When I change variable type to CWSTR, an error occurs with the Line Input# statement.
It wants a string variable.
Logged

Jos Roca

  • Guru Member
  • *****
  • Posts: 3325
Re: Listbox - unicode items
« Reply #7 on: April 07, 2021, 02:28:02 PM »

If the file is utf-8, you have to read it using ansi strings and then convert it to ansi or unicode, since a listbox (or any other Windows control) doesn't understand utf-8.
« Last Edit: April 07, 2021, 02:31:07 PM by Jos Roca »
Logged

Bumblebee

  • Junior Member
  • **
  • Posts: 75
Re: Listbox - unicode items
« Reply #8 on: April 07, 2021, 03:55:56 PM »

WStr() function will not convert strings read with Line Input#
It does work with literal strings.
Code: [Select]
#include "Afx\CWStr.inc"
dim a as string
dim b as CWSTR
a = "lgance"
b = "lgance2"
print a
print b
print wstr(a)
b = wstr(a)
print b
print "- read utf8 file -"
a=""
open "test.txt" for input as #1
line input #1,a
print a
b = a
?b
?wstr(a)
?wstr(b)
close
sleep
end
Logged

Jos Roca

  • Guru Member
  • *****
  • Posts: 3325
Re: Listbox - unicode items
« Reply #9 on: April 07, 2021, 04:57:37 PM »

> WStr() function will not convert strings read with Line Input#

Of course not. WSTR will convert ASCII to UNICODE, not UTF-8 to UNICODE.

You can either use the Windows API function MutibyteToWideChar or...

DIM cws AS CWSTR = CWSTR(<UTF-8 string>, CP_UTF8)

e.g.:

DIM s AS STRING = "José Roca"   ' My name in UTF-8
DIM cws AS CWSTR = CWSTR(s, CP_UTF8)
print cws
« Last Edit: April 08, 2021, 04:49:21 AM by Jos Roca »
Logged

Bumblebee

  • Junior Member
  • **
  • Posts: 75
Re: Listbox - unicode items
« Reply #10 on: April 07, 2021, 10:35:33 PM »

I don't understand any of this, but it works. Thanks!
Logged

Jos Roca

  • Guru Member
  • *****
  • Posts: 3325
Re: Listbox - unicode items
« Reply #11 on: April 08, 2021, 04:56:52 AM »

You should learn the differences between ASCII, ANSI, UTF-8 and UNICODE.

Bumblebee

  • Junior Member
  • **
  • Posts: 75
Re: Listbox - unicode items
« Reply #12 on: April 08, 2021, 10:47:24 PM »

Code: [Select]
Dim z as String
~
Line Input #1, z
z = CWSTR(z,CP_UTF8)

You said that accented characters are not unicode, so this works.
When z is written to a text file, the file is ANSI.

What could I do if there were Cyrillic characters in the UTF-8 source file?
Logged

Jos Roca

  • Guru Member
  • *****
  • Posts: 3325
Re: Listbox - unicode items
« Reply #13 on: April 09, 2021, 06:47:00 AM »

An UTF-8 file can't contain cyrillic characters, it has to be UTF-16.

If you need to read files with unicode content, you can't use Line Input. As I said, FB support for unicode is weak. You can use my class CTextStream: https://github.com/JoseRoca/WinFBX/blob/master/docs/File%20Management/CTextStream%20Class.md

Bumblebee

  • Junior Member
  • **
  • Posts: 75
Re: Listbox - unicode items
« Reply #14 on: April 09, 2021, 10:30:29 AM »

I took the Cyrillic characters you posted and saved them in Notepad. It says the file is UTF-8.
Is this dependent on my language settings?

Code: [Select]
#include "Afx\Cwstr.inc"
dim a as string
dim b as CWSTR
'cyrillic characters encoded as utf-8
open "test.txt" for input as #1
line input #1,a
close
print a
b = a
print b
print cwstr(a,cp_utf8)
print cwstr(b,cp_utf8)
sleep
end

Ouput in terminal window:

╨╨╡╤╨│╨╡╠╨╣ ╨╨╡╤╨│╨╡╠╨╡╨▓╨╕╤ ╨╤╨╛╨║╨╛╠╤╤╨╡╨▓
Серге́й Серге́евич Проко́фьев
Серге́й Серге́евич Проко́фьев
Серге́й Серге́евич Проко́фьев

So maybe it would work.
When I was working on my file backup program, CWSTR was able to handle every filename, including those that had korean characters. However, I wasn't writing/reading those names from text files. Everything was done within CWSTR arrays.
Logged
Pages: [1] 2