The hourglass is meaningless when Windows has marked the program as non-responsive. If a program is stuck in an infinite loop it will show an hourglass too, how is a user going to know the difference? When I see a program labeled as "Not Responding" I give it a few seconds to sort itself out and then I force kill it. This has been a solved problem for decades now, it's terrible user interface design, and it points to piss poor coding internally.
Do better then talk? Here is the code to deduplicate a TStringList while updating a progress bar and preventing your app from being labeled as non-responsive:
OrigStringList.LoadFromFile('names.txt');
DedupStringList.Sorted := True;
DedupStringList.Duplicates := dupIgnore;
for i := 0 to OrigStringList.Count - 1 do
begin
DedupStringList.Add(OrigStringList[i]);
if i mod 1000 = 0 then
begin
ProgressBar1.StepIt;
Application.ProcessMessages;
end;
end;
Brain dead simple, tested on a Pentium Dual Core T4500 laptop with 2GB of memory, sample data set was 250 unique first and last names randomly distributed over 6,000,000 entries creating an 84 MB text file. In a simple console application, the file was loaded to OrigStringList and deduplicated by transferring the to another TStringList with TStringList.Duplicates set to dupIgnore using TStringList.AddStrings(). This took 43 seconds and resulted in DedupStringList containing the original 250 unique strings. For the next test the same dataset was used in a graphical application using the code above to deduplicate the list. It took 46 seconds to process, updated the progress bar, and the application remained responsive allowing the window to be moved, minimized and maximized, as was not labelled "Not Responding" by the Task Manager. That can be further spread up by increasing the number of iterations between calls to Application.ProcessMessages and how often the progress bar is updated, that can be adjusted to the number of entries be processed, the type of progress bar being updated, etc etc. Using this method you'll want to either disable the form, or better yet, disable the individual controls that you don't want users clicking during the processing. You'll definitely want to disable the De-Deplicate button to prevent inpatient users from clicking it multiple times and stuffing the message queue.
That's the Amateur Hour was of doing this, all simple intrinsic methods and minimal work on your part and still performant enough. Especially as you advertise your program as being "multi-threaded" it is just embarrassing that your UI locks during processing. That leads me to the better method, doing your processing in a background thread and updating your UI with TThread.Synchronize, but the above method is good enough for a simple list processing app. Your app ran on my desktop, an Intel i7 4790 with 16GB memory and was still not responding after MINUTES of processing. This is absolutely ridiculously under-performant when we're talking about megabytes of data, even with fancy Delphi string processing. You constantly harp on the speed of in-browser ad blocking, so ask yourself this: if these extensions (which I believe are written in JavaScript!) can run through their blacklists on every page load in seconds, if they can install and update their lists in seconds, how come your app can't process a few million entries is less than five minutes using native code? The answer is very basic knowledge of data structures and algorithms on behalf of the developers.
