View Single Post
Old 05-16-2024, 05:07 PM   #4
dandman
Enthusiast
dandman is clearly one to watchdandman is clearly one to watchdandman is clearly one to watchdandman is clearly one to watchdandman is clearly one to watchdandman is clearly one to watchdandman is clearly one to watchdandman is clearly one to watchdandman is clearly one to watchdandman is clearly one to watchdandman is clearly one to watch
 
Posts: 29
Karma: 10545
Join Date: May 2024
Device: none
as far as i can tell, every record i added to the queue was displayed to the user,

there was just one strange behavior worth mentioning:
after every call to url using
PHP Code:
self.browser.open_novisit() 
the result_queue would be emptied, so i had to back it up before the call to the open_novisit and restore it after the call, like this:

PHP Code:
            # Create a separate queue for downloaded metadata
            
downloaded_metadata_queue Queue()

            
# Before calling open_novisit(), save downloaded metadata
            
while not result_queue.empty():
                
item result_queue.get() #get also remove the item
                
downloaded_metadata_queue.put(item)
            
            
# call the URL
            
response self.browser.open_novisit(request)
            
            
# Merge the downloaded metadata back to result_queue after the call
            
while not downloaded_metadata_queue.empty():
                
item downloaded_metadata_queue.get()
                
result_queue.put(item
dandman is offline   Reply With Quote