• Welcome to PlanetSquires Forums.
 

list box limits

Started by raymw, May 08, 2018, 05:41:31 PM

Previous topic - Next topic

raymw

I'm trying to read fairly large ASCII text files, about 2MB, and 500,000 lines or so, some will be much larger. Each line will be something like 'n677767 a453.0009 b 78.6534 c-98.5555' I need to be able to perform various manipulations on the values following each letter, then save the resultant file. I can load the file into windows notepad, and quickly manually scroll anywhere in the file - (earlier notepad versions had a size limit, afaik).

My initial intention was to read the file, a line at a time, and add into a listbox. This works, albeit quite slow, but once the list box is populated, the scroll arrows are too slow, makes it rather useless for my purpose. Any suggestions as to getting the scrolling to perform? My intention was then to parse each line in the list, and separate  into individual lists of values for a, b and c, etc. and perform the modifications on each list, then recombine and save as new file.

I'm guessing, instead of putting values into a list box (delays in showing values on screen) it be quicker to not use a list box, but just a list, and somehow use a slider to present the required screen view of part of the list.

Best wishes,

Ray


Paul Squires

You could use a virtual list box or virtual listview. This may be a little too complex for your needs though. You read your data into array and the virtual control only reads and displays the data it needs to display on the screen. They are extremely fast however I don't think I've seen one for freebasic. There are several powerbasic source codes available (eg. Borje's PBVlist) but it would take someone time to convert it to freebasic.

Paul Squires
PlanetSquires Software
WinFBE Editor and Visual Designer

Paul Squires

Lol, I stand corrected. :)  Jose has pointed to a port done by Jim. Nice.
Paul Squires
PlanetSquires Software
WinFBE Editor and Visual Designer

raymw

#4
Thanks, I'll have a look, and most likely will have to rethink my modus operandus. iirc, some time ago, when I was playing with c#, I was able to open a file in notepad by using some sort of system call. I can't remember if I was able to get at the data from the notepad file. But, back then notepad was restricted as to the size of file it could handle.
I'm guessing, handling larger files, etc., I'd be better off using 64 bit free basic/firefly.

I've now had a chance to look at the vlist.inc, etc. seems it will do the job, played with the demo program, generated a list of one and a half million lines similar to my requirements, scrolls fast enough, etc. Think I'll need to set variable speed scroll bars. It will take me a while to get my head around how to use it in my own program, however - a lot of statements I've never used.

raymw

I have thoroughly played with Jim's example code, and I think it will be fine for displaying/handling my files. At the moment, I want to use firefly to generate the gui part of things, but I'm completely stumped on how to merge in the virtual list box, and I think I'll need a number of them. I am not into the programmatically arranging of components, much prefer the visual approach of firefly, etc. Can I somehow equate the virtual list box to the firefly listbox, or will it be more expedient to place them by manually coding, (copying the bits of the create code from the example, and the rest of the form layout in firefly?) Any help much appreciated, but I'll need to take it in 'baby steps' I expect

Paul Squires

#6
Yes, you will have to do this manually. #Include Jim's inc file and then copy the bits of code from his .bas file that you need in order to make it work. I would put the following listbox creation code in the form's WM_CREATE message handler:

   VL_Int
   
   hVList1   = CreateWindow ("FBVLIST",_      'window class name
                       BYVAL NULL,_          'window caption
                       FBVLISTSTYLES,_       'window style
                       4,4,300,300,_         'initial Position
                       hWnd,_                'parent window handle
                       cast(hMENU,IDC_VLIST),_   'window menu handle
                       hInstance,_               'program instance handle
                       BYVAL NULL)

I expect that you will encounter High DPI scaling issues if you use the code on systems where the default 96 dpi is not used. It doesn't look like the code scales up the incoming size parameters in the CreateWindowEx call.
Paul Squires
PlanetSquires Software
WinFBE Editor and Visual Designer

raymw

Thanks Paul. If it was easy, everybody would be doing it...

Paul Squires

I am attaching a sample FireFly FreeBasic project that shows how to use the control within FireFly. (I use two CustomControls and tie the virtual listboxes to them so you can visually move the controls around the screen, etc).
Paul Squires
PlanetSquires Software
WinFBE Editor and Visual Designer

raymw

Thanks Paul. that does it fine.

raymw

Thanks to your help and suggestions, I'm now able to process the files, longest so far was 3 million lines. It would be nice if I could use a slider or scroll bar to get to the lines I want to inspect. I can use a line number to select a particular line, but then I sometimes want to be able to have a look at the adjacent 50 or so lines, and the scroll bars at the listbox side do not have fine enough resolution for the large arrays. I see there is an up/down control in firefly, but I'm not sure if that will do what I need. I have no real idea on how it works, and my 'suck it and see' test approach crashes the program

Function FRMMAIN_UPDOWN1_CUSTOM ( _
                                ControlIndex  as Long,  _     ' index in Control Array
                                hWndForm      as HWnd, _      ' handle of Form
                                hWndControl   as HWnd, _      ' handle of Control
                                wMsg          as UInteger,  _  ' type of message
                                wParam        as WPARAM, _    ' first message parameter
                                lParam        as LPARAM   _   ' second message parameter
                                ) as Long
        Dim pp as Long                       
pp= FF_UpDown_GetPos( hwnd_frmmain_updown1)
?pp
   Function = 0   ' change according to your needs
End Function


I am looking for a control that will step through a list, at a speed/step size based on the cursor position in the control, i.e. the further away from the centre position of control, the faster the window scrolls. Is that how the updown control works?

Pierre Bellisle

#11
"have a look at the adjacent 50 or so lines"

If I understand you correctly, all you have to do is to click in the page up/down free space between the thumb and the line up/down button.

Also, for a custom control listbox or a subclassed listbox, you may use the keyboard, like CTRL-Click to do custom scroll.

You could also add more stand alone scroolbar to do medium and fine scrolling...

raymw

Hi Pierre, thanks for your suggestion. What I've simply done, until I can figure out if the updown control will be better, is create a column of 8 buttons, the two centre ones moves the selected line ten lines, the next two 100 lines, the next pair 1000, the end ones 10000 lines. Not perfect, but combined with the existing scroll bar it is good enough for me, unless a better idea comes up. I could vary the step size, based on the size of the file, or other parameters- not much point in stepping 10000 lines if the file is only 500 lines long!

raymw

I've been honing what I've been trying to do, it is now sort of usable, but a couple of things I'd like to be able to change.
1) Is it possible to show a multi-line tool tip? In some instances, 128 chars does not describe what the control does, and I'm sure that in a few month's time, I'll have forgotten
2) When I select a line in the list box with the cursor, it is highlighted with a blue background  and white text, and is quite readable. However, when a line is selected programmatically the line is highlighted  with white text on a grey background, and is not as easy to read.  It would be nice to be able to change the grey to something more contrasting.

Paul Squires

#2 the listbox probably needs keyboard focus. Probably do a SetFocus(hwndListBox) after you load the items. This may or may not work depending on if the focus gets stolen to a different control after you load the items. If it is done on program startup (when your form is loading) you can put it at the end of the the WM_CREATE handler (worst case you may have to do a PostMessage at end of WM_CREATE and set focus in the custom handler based on a user defined message - there are examples in this forum on that approach).
Paul Squires
PlanetSquires Software
WinFBE Editor and Visual Designer