Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - afrocuban

Pages: [1] 2 3 4 5 6 ... 32
1
PVD Python Scripts / Re: PVD Selenium MOD v4 IMDb Movie, People and FilmAffinity Scripts
« on: October 26, 2025, 10:55:50 pm »
Great. I have fixed Movie Connections, and full AKA. Also fixed Selenium_Chrome_Movie_Additional_pages_v4.py so now it clicks on "See more" and similar objects, again. I will not upload next iterations because I want to fix whole Base page function first, and  for me it looks that IMDb makes it harder and harder to scrape data, since for now, I'm often seeing errors IMDb killing connections to host when loading additional pages with selenium, so I have to investigate selenium and python options to mimic human browsing as best as possible.

If you still want to have current versions of my scripts, let me know so I could upload them with the manual how to get data I'm getting at the moment.

2
PVD Python Scripts / Re: PVD Selenium MOD v4 IMDb Movie, People and FilmAffinity Scripts
« on: October 24, 2025, 10:47:52 pm »
I have optimized Selenium_Chrome_Base_page_v4.py script so now downloading of the IMDb main page should be dramatically faster - around 18 seconds on my computer.

3
PVD Python Scripts / Re: PVD Selenium MOD v4 IMDb Movie, People and FilmAffinity Scripts
« on: October 24, 2025, 01:33:43 am »
Quote from: Ivek23 on October 21, 2025, 07:36:21 am

For the Episode List page, it would be necessary and even better if a script was made specifically for it.

Episode list works totally fine for me, even now. It would be probably impossible for me to create new script. We can look at it like this: Episode list has it's own script: it's Selenium script "Selenium_Chrome_IMDB_Episode_List_page_v4.py" which produces txt file with all episodes, and .psf script just scrapes that txt file, so I am not sure what we could achieve with additional psf script whose purpose would be only to scrape txt file.

Anyway, I have corrected FilmAffinity script, and made some improvements in IMDb Movie scripts (for example, storyline section  dynamic load scraping is fixed). I have improved FilmAffinity script speed enormously! In order this script to work properly and fast you need:
1. To install python 3.12+
2. in a cmd to install psutil with
Quote
pip install psutil
in order to hopefully prevent selenium hangups when html elements aren't found on the page.
3. To download and overwrite scripts I'm uploading in this post.
4. In a script configurator to deselect Reference page. PVD will restart. Then select "Studio" and "Description" and all others you want and do not restart PVD. Try to import data, and now you should get director, cast, tagline and some other original and related custom fields scraped from the Main page. If you don't get cast, tagline and director, restart PVD manually and try again to import. I am not sure about this second restart, so try both. One will work for sure.

I will not provide support for these scripts until I finish, because I know they still don't fully work. I just want to share with you same amount of data I'm getting at the moment when some significant improvement is done. In a pictures you can get the sense of what i'm getting now with IMDB script.

Once I finish IMDb script we will test and correct it together.

4
Scripts and Templates / Re: Export template: HTML List (English + German)
« on: October 19, 2025, 02:40:46 pm »
Quote from: svenne on April 09, 2025, 10:40:20 pm
Hello to all of you,

I haven't been here for a long time. Work and especially family have completely taken over. But I'm still using PVD myself and it's nice to see there is still at least some life in here. Big thanks to Ivek23, I think. It is a great pity that Nostra has put PVD on ice. I really hope he is doing well despite all the current upheavals in the world and that his reasons behind his absence are of a similarly positive nature as those for mine...

Some time ago, I've noticed that large parts of the JavaScript in the HTML frames export template no longer work (at least if exported files are local and not hosted on a domain). The reason is the "same origin policy" that became general standard and that browsers may treat all file:/// URIs as unique origins. In this case, JavaScript in iFrames is no longer allowed to directly access parts of their parents and vice versa. In the exported database, this affects filters, the search function and most of the extras added by pimpHTMLframesExport.exe (which can be found in the subfolder \pimpHTMLframesExport after the export). I fixed this, but tested only in Firefox. The new export templates are attached.

All the kindest wishes to those still here!

Thanks svenne. It's allways great to hear family takes first pace. Updated template works flawlessly. I wanted to ask you if you're willing to help with lastDino's template too, if that is viable for you. Thank you in advance!

5
PVD Python Scripts / Re: PVD Selenium MOD v4 IMDb Movie, People and FilmAffinity Scripts
« on: October 19, 2025, 12:51:51 pm »
Quote from: afrocuban on March 23, 2025, 01:55:13 am
Soon, /reference page will be changed too. I know because I got popups offering me to peek to a new "Reference" page. So, until that happen, I will not update scripts, because both pages will share the same code again, and it will be easier to change.


Hello to all. As I already said in March, now we all know that this happened. I was busy meanwhile and had no time to deal with it. There are changes across many imdb pages, but reference page is the culprit. I will need couple of months to fix it, since you all know that I am not a programer, and I have to remind my self about everything, especially about special cases. For now, you can pull up significant portion of data without reference page (unfortunately, not the whole cast). To do that and if you are using my scripts, open Script configurator, uncheck to download Reference page and PVD will restart. Then, in "Set Overwrite options...", check all the data you would want to download (Studio, etc..) and restart PVD. Then run the script and this would be what you can get at the moment.

After March, I had some minor adjustments to the scripts and started to work on a Reference page, so I'm uploading them so you could get the same amount data as me. Backup your existing scripts, then overwrite them with these, all to "Scripts" folder. Reminder: these scripts are just starting point to fix them, but they should get you more data comparing to March scripts could get you now, so please do not ask for the support for these scripts. I know they don't fully work.


What should be promising for you is that I'm not planning to abandon using PVD, so I will for sure fix the scripts at some point, so please be patient: if long time no see me, that just means I also haven't fixed the scripts, and I'm working on them. Meanwhile, just add your movies to PVD and later you will update with full data. That is exactly what I've been doing recently.

Best regards

6
PVD Python Scripts / IMDb ALL-IN-ONE SCRIPT
« on: March 23, 2025, 01:55:13 am »
IMPORTANT!!!

A few hours ago, IMDb completely changed /fullcredits page html layout, so that page doesn't work any more. Soon, /reference page will be changed too. I know because I got popups offering me to peek to a new "Reference" page. So, until that happen, I will not update scripts, because both pages will share the same code again, and it will be easier to change. For now I made a quick fix everything to work if you check the options in Configurator as I suggested earlier. In addition you have to check "Download the Cast or Credit (text only) provider page to retrieve the full information. Or else, only the info from the main movie page will be downloaded." option and to download fullcredits page too!!! This should work until /reference page changes, or any other page changes meanwhile.

And it happened just when I finished ''all-in one script" while successfully doing final tests. Here's the pack.


Quote
So, with one IMDb Script you get all movies, Series, episode list, and then you apply the same script for episodes.


Also, new search window introduced, with different types of search and countdown of 10 seconds defaulted to "general" search.

It took only 600 additional lines comparing to Movie script, including a lot of commented out lines, and one simple python script to get all of this.

Extract and overwrite existing scripts with this pack.


I will soon start to re-birth AllMovie and RottentTomatoes scripts. I will not revive any other script.

7
PVD Python Scripts / Re: Selenium Imdb Series and Episodes Scripts discussion
« on: March 21, 2025, 09:52:07 pm »
Thanks!

8
PVD Python Scripts / Re: PVD Selenium MOD v4 IMDb Movie Script Confusion
« on: March 21, 2025, 09:50:34 pm »
Thanks Ivek.


here's the assesment of these 3 snippets:


The only thing from the first 2 snippets  I see is that commented out lines are deleted from the original. I compared everything in Notepad++:

Quote
      //MovieURL := 'http://www.imdb.com' + TextBetWeenFirst(ItemList, '", "url":"', '", "name":"');
Quote
      //LogMessage('Function ParsePage_IMDBMovieBASE -   *   Get result url 1: ' + MovieURL + ' | |');

Quote
         // If titleValue = '' then titleValue := TextBetWeenFirst(ItemList, '<h1 class="long">', '<'); // Strings which opens/closes the data. WEB_SPECIFIC



These lines, which were already commented out, are just deleted as "improvement" but they don't influence anything anyway, so I am not sure what improvement is there? With or without them, everything works anyway.

Also, the third snippets works too, it's just the other way to achieve the same. There are so many ways to achieve the same goal, and I choose one, that works too.

I hope to see what doesn't work actually in the scripts.



9
PVD Python Scripts / Re: Selenium Imdb Series and Episodes Scripts discussion
« on: March 21, 2025, 01:35:44 pm »
Question:
Which script creates seasons in PVD?

10
PVD Python Scripts / Re: PVD Selenium MOD v4 IMDb Movie Script Confusion
« on: March 21, 2025, 08:27:17 am »
For my personal use, I will disable all the duplicates of the custom fields once I set all the scripts. We were foolish we were allowed to ask to have like 6 custom fields for Aspect ratio, for example. I just brought them all so users can decide which one they want. If some are empty, it's either they don't exist for the given movie, or they are set by Configurator not to be brought. Logic of the script is extremely complex. But let's see.

11
PVD Python Scripts / Re: Chrome Selenium v4 of Movie and People scripts progress
« on: March 21, 2025, 12:07:29 am »
While we are waiting Ivek to decide if I will continue to share my script, I have integrated Series script into Movie Script and upgraded search with the coundtdown for the default "general" search. In the screenshoot you can see search window option, as well as series imported with movie script

12
PVD Python Scripts / Re: Chrome Selenium v4 of Movie and People scripts progress
« on: March 20, 2025, 11:17:27 pm »
Thanks, Ivek. As usual, if you could be more specific, it would be helpful.

13
PVD Python Scripts / Re: Selenium Imdb Series and Episodes Scripts discussion
« on: March 17, 2025, 09:40:34 am »
Thanks for the explanation. I'm looking for ways how to automate this without limiting on number os seasons and episodes...

14
PVD Python Scripts / Re: Selenium Imdb Series and Episodes Scripts discussion
« on: March 16, 2025, 05:50:56 pm »
Thanks ivek. I already looked at those scripts of course, but what I don't get is is how PVD knows when first season is finished. When Function ParsePage_IMDBMovieSEASON1 stops with AddEpisode, and Function ParsePage_IMDBMovieSEASON2 starts?

15
PVD Python Scripts / Selenium Imdb Series and Episodes Scripts discussion
« on: March 16, 2025, 01:01:50 pm »
Quote from: Ivek23 on March 16, 2025, 10:00:47 am
Quote from: Ivek23 on March 16, 2025, 07:47:18 am
I must have made a mistake, I just didn't notice it (I still have a lot of things to sort out for my mother's passing, so some details are missing and I don't notice them). I'll fix that and see if it works.

I fixed it now and it works perfectly.

Great! Enjoy it!

May I ask you a question? Can you describe the flow of adding series, seasons and episode links? Which script does which task? I'm almost done integrating series script into movie script, but I am stuck at the moment what generates seasons and what generates  episodes in each season, what provides links to the episodes and so on. Thank you in advance.

16
PVD Python Scripts / Re: Firefox Selenimu Script Discussion
« on: March 15, 2025, 10:57:33 pm »

Quote from: Ivek23 on March 15, 2025, 08:44:54 pm
Interesting, I also did a test with Chromedriver (without using the Chrome browser), and it works great there.


I am so happy to hear it works!

Quote from: Ivek23 on March 15, 2025, 08:44:54 pm
Here are the Scripts adapted for firefox so that you can see my adaptation and maybe errors or I did something wrong or did not change anything.


The script looks mostly fine, but there is an issue in the following lines:

Quote
firefox_options_options.add_argument("--headless")  # Running Firefox in headless mode
firefox_options_options.add_argument(f"--lang={language_code}")
firefox_options_options.add_argument("--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36")
firefox_options_options.add_argument("--enable-unsafe-swiftshader") # Add this flag to use unsafe SwiftShader

Here, firefox_options_options is used, but the variable should be firefox_options. The correct lines should be:

Quote
firefox_options.add_argument("--headless")  # Running Firefox in headless mode
firefox_options.add_argument(f"--lang={language_code}")
firefox_options.add_argument("--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36")
firefox_options.add_argument("--enable-unsafe-swiftshader") # Add this flag to use unsafe SwiftShader

Once this change is made, the script should work as expected. Other than that, the syntax looks correct! Just make sure you have the necessary dependencies installed and the correct path to geckodriver.

17
PVD Python Scripts / Re: Firefox Selenimu Script Discussion
« on: March 15, 2025, 07:37:21 pm »
Quote from: Ivek23 on March 15, 2025, 10:11:16 am

No, I didn't rename Selenium_Chrome_Base_page_v4.py or the other python files, because it would be too time-consuming to do it everywhere in the .psf files. Everything I did in all the python files, I only changed the settings on firefox and geckodriver.
When the driver is called in python script, is it called as chrome, or gecko? It would be good to share you script so we could visually see how you adjusted it.

Quote

That's all that was written in python_script_base_page.txt.
Quote
2025-03-15 10:08:08,399 - DEBUG - Starting the Python script.
2025-03-15 10:08:08,403 - DEBUG - Starting new HTTP connection (1): ipinfo.io:80
2025-03-15 10:08:08,641 - DEBUG - http://ipinfo.io:80 "GET /country HTTP/1.1" 200 27
2025-03-15 10:08:08,642 - DEBUG - Country code: SI, Language code: sl


This why I suspect driver isn't called at all... Try to test the script from the cmd and you will get more informative response. For the title search:

Quote
pyhton FullPathToTheScript titleIMDb "10 Things...." (with the double qoutes, or single quotes, it depends on your setting, try them both)

for the main page:

Quote
pyhton FullPathToTheScript "MovieURL" "FullPathToThe\downpage-UTF8_NO_BOM.htm" (with the double qoutes, or single quotes, it depends on your setting, try them both)

18
PVD Python Scripts / Re: Firefox Selenimu Script Discussion
« on: March 15, 2025, 08:31:48 am »
Ok, that's better. Let's move debugging to the other topic, you may call it Firefox Selenium?  Several ideas:
1. Did you rename any files, in this case did you rename Selenium_Chrome_Base_page_v4.py? If so, rename everywhere in the .psf too.
2. What says in the correspondent base.log file in \Tmp folder?

19
PVD Python Scripts / Re: Chrome Selenium v4 of Movie and People scripts progress
« on: March 15, 2025, 12:09:36 am »
I am sorry to hear that. If you could be a bit more specific maybe I'd get an idea what it might be.

In the meantime, I have started to work on upgrading Selenium Chrome search script to be "one for all". Now you can chose between different title types as I grouped them, plus when importing for example series and movies at the same time with "Tools->Scan folders...", now you can use "general" search. I will try to merge IMDb Movie and Series script, and hopefully Episodes at the end. There aren't much differencies at the first glance analyzing them

20
PVD Python Scripts / Re: Firefox Selenimu Script Discussion
« on: March 14, 2025, 12:04:39 pm »
Quote from: Ivek23 on March 14, 2025, 11:56:15 am
Thanks for the comprehensive explanation for Firefox browsers.

Thank you very much.

You are more than welcome, Ivek. I never tried it, so I am not sure at all how Firefox would download pages (clicking "See more" pages, "Storyline" sections and other), and if final html code would be the same as downloaded with Chrome, so it might be frustrating to realize that there are differences actually in scraped hmtls with either.

P.S. In people script, I brought back career option to base function too, so just make sure the proper switch (ShouldParseCareer) is set not to parse it with bio function.

Pages: [1] 2 3 4 5 6 ... 32
anything