Personal Video Database

Just in case other people get this to fix it in Selenium_Chrome_Movie_Additional_pages_v4:

after driver.get(download_url)

i added:

time.sleep(random.uniform(8, 12))

This change does not work because it blocks the download of Additional pages.

It is important to note that you must have the latest version of chromedriver.exe for this to work. Chromedriver always needs to be updated to the latest version, this is a prerequisite for all scripts to work.

Somehow, despite my health problems, I managed to check how the latest IMDb Movie, Allmovie and Rottentomatoes v4 Scripts work. They work fine (with some cosmetic errors when transferring information)  provided that you use the latest chromedriver.exe program in what I mentioned a little higher up.

I didn't check the other scripts.

Thank you for all the effort put into creating all the scripts. As for me, I currently have quite a few health problems, so I am currently less present on the forum and currently because of this I am using the PVD program less or very little and testing scripts and the like.

My wish is that you would still help update all the scripts.

As for other users, I assume that some people find it difficult to use or install the python program on their computer, because they may also be less skilled in using such programs.

To clarify, I myself do not know many things about programming, because I am self-taught and have never had courses in using Windows and programming.

Hey, Ivek. Thanks. Can you please post examples or imdb links where those matters and my code doesn't work? I just can't grasp just by looking at the code. Thanks.

To be clear, these parts work correctly, my fixes to some of the code are only cosmetic in nature. Below are examples of what I had in mind.

Before

<link url="https://www.imdb.com/title/tt0147800/parentalguide/#contentRating">Content Ratings Summary:          </link><link url="https://www.imdb.com/title/tt0147800/parentalguide/#contentRating">Content rating (5)   •   </link><link url="https://www.imdb.com/title/tt0147800/parentalguide/#nudity">Sex & Nudity (10)   •   </link><link url="https://www.imdb.com/title/tt0147800/parentalguide/#violence">Violence & Gore (7)   •   </link><link url="https://www.imdb.com/title/tt0147800/parentalguide/#profanity">Profanity (   •   </link><link url="https://www.imdb.com/title/tt0147800/parentalguide/#alcohol">Alcohol, Drugs & Smoking (5)   •   </link><link url="https://www.imdb.com/title/tt0147800/parentalguide/#frightening">Frightening & Intense Scenes (1)   •   </link><link url="https://www.imdb.com/title/tt0147800/parentalguide/#certificates">Certifications (49)</link>

and after

<link url="https://www.imdb.com/title/tt0147800/movieconnections/">Connections:</link>     •     <link url="https://www.imdb.com/title/tt0147800/movieconnections/#featured_in">Featured in (134)</link>    <link url="https://www.imdb.com/title/tt0147800/movieconnections/#features">Features (1)</link>    <link url="https://www.imdb.com/title/tt0147800/movieconnections/#followed_by">Followed by (2)</link>    <link url="https://www.imdb.com/title/tt0147800/movieconnections/#referenced_in">Referenced in (59)</link>    <link url="https://www.imdb.com/title/tt0147800/movieconnections/#references">References (</link>    <link url="https://www.imdb.com/title/tt0147800/movieconnections/#spin_off">Spin-off (1)</link>    <link url="https://www.imdb.com/title/tt0147800/movieconnections/#spoofed_in">Spoofed in (2)</link>    <link url="https://www.imdb.com/title/tt0147800/movieconnections/#spoofs">Spoofs (1)</link>    <link url="https://www.imdb.com/title/tt0147800/movieconnections/#version_of">Version of (58)</link>

Before

<link url="https://www.imdb.com/title/tt0147800/movieconnections/">Connections:                               </link><link url="https://www.imdb.com/title/tt0147800/movieconnections/#featured_in">Featured in (134)   •   </link><link url="https://www.imdb.com/title/tt0147800/movieconnections/#features">Features (1)   •   </link><link url="https://www.imdb.com/title/tt0147800/movieconnections/#followed_by">Followed by (2)   •   </link><link url="https://www.imdb.com/title/tt0147800/movieconnections/#referenced_in">Referenced in (59)   •   </link><link url="https://www.imdb.com/title/tt0147800/movieconnections/#references">References (   •   </link><link url="https://www.imdb.com/title/tt0147800/movieconnections/#spin_off">Spin-off (1)   •   </link><link url="https://www.imdb.com/title/tt0147800/movieconnections/#spoofed_in">Spoofed in (2)   •   </link><link url="https://www.imdb.com/title/tt0147800/movieconnections/#spoofs">Spoofs (1)   •   </link><link url="https://www.imdb.com/title/tt0147800/movieconnections/#version_of">Version of (58)</link>

and after

<link url="https://www.imdb.com/title/tt0147800/parentalguide/#contentRating">Content Ratings Summary:</link>          <link url="https://www.imdb.com/title/tt0147800/parentalguide/#contentRating">Content rating (5)</link>   •   <link url="https://www.imdb.com/title/tt0147800/parentalguide/#nudity">Sex & Nudity (10)</link>   •   <link url="https://www.imdb.com/title/tt0147800/parentalguide/#violence">Violence & Gore (7)</link>   •   <link url="https://www.imdb.com/title/tt0147800/parentalguide/#profanity">Profanity (</link>   •   <link url="https://www.imdb.com/title/tt0147800/parentalguide/#alcohol">Alcohol, Drugs & Smoking (5)</link>   •   <link url="https://www.imdb.com/title/tt0147800/parentalguide/#frightening">Frightening & Intense Scenes (1)</link>   •   <link url="https://www.imdb.com/title/tt0147800/parentalguide/#certificates">Certifications (49)</link>   •

IMDB_Movie_[EN][Selenium]-v4.psf

Fixed part of the code in Function ParsePage_IMDBMovieCONNECTIONS

Previous
// Get (CF~Connections~) info   
If Pos('>Connections<', HTML) > 0 Then
Begin
LogMessage('===Function ParsePage_IMDBMovieCONNECTIONS - Starting Connections Parsing ===');

curPos := Pos('>Connections<', HTML);
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - curPos for "Connections" section found at: ' + IntToStr(curPos));
If Pos('It looks like we don'+Chr(39)+'t have any Connections for this title yet.', HTML) > 0 Then
Begin
Category1 := 'No Connections';
Category3 := '';
AddCustomFieldValueByName('Connections', Category1);
AddCustomFieldValueByName('Connect', Category3);
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - No Connections found for this title');
End Else Begin
// Extract the relevant section for categories
If PosFrom('<span class="ipc-simple-select__container" data-testid="jumpTo">', HTML, curPos) > 0 Then Begin
curPos := PosFrom('<span class="ipc-simple-select__container" data-testid="jumpTo">', HTML, curPos);
endPos := PosFrom('</select></span></span>', HTML, curPos);
Category2 := Copy(HTML, curPos, endPos - curPos);
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Extracted Category2 section: ' + Category2);

// Parse the options and category names
curPos := 1;
Category1 := '';
While PosFrom('<option', Category2, curPos) > 0 Do
Begin
curPos := PosFrom('<option', Category2, curPos) + Length('<option');
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - curPos at option tag: ' + IntToStr(curPos));

optionValue := TextBetween(Category2, 'value="', '">', False, curPos);
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Extracted optionValue: ' + optionValue);

categoryName := TextBetween(Category2, '">', '</option>', False, curPos);
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Extracted categoryName: ' + categoryName);

// Format the category link
Category1 := Category1 + '<link url="' + MovieURL + optionValue + '">' + categoryName + '   •   </link>';
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Formatted and added category link: ' + '<link url="' + MovieURL + optionValue + '">' + categoryName + '   •   </link>');

// Move to the next position
curPos := PosFrom('</option>', Category2, curPos) + Length('</option>');
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Moved curPos to next option tag: ' + IntToStr(curPos));
End;

// Remove the trailing "   •   " if it exists and keep the last </link> intact
If Copy(Category1, Length(Category1) - 13, 7) = '   •   ' Then
Begin
Category1 := Copy(Category1, 1, Length(Category1) - 14) + '</link>';
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Removed trailing "   •   " and kept last </link> in Category1');
End;

LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Final Category1 before combining: ' + Category1);

// Combine and format the final result
Category1 := '<link url="' + MovieURL + '">Connections:                               </link>' + Category1;
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Combined and formatted final Category1: ' + Category1);

// Store the result in the custom field
AddCustomFieldValueByName('Connections', Category1);
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Stored result in (CF~Connections~)');

LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Final Category1 stored: ' + Category1);
LogMessage('===Function ParsePage_IMDBMovieCONNECTIONS - Finished (CF~Connections~) Parsing ===');
End;
Fixed
// Get (CF~Connections~) info   
If Pos('>Connections<', HTML) > 0 Then
Begin
LogMessage('===Function ParsePage_IMDBMovieCONNECTIONS - Starting Connections Parsing ===');

curPos := Pos('>Connections<', HTML);
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - curPos for "Connections" section found at: ' + IntToStr(curPos));
If Pos('It looks like we don'+Chr(39)+'t have any Connections for this title yet.', HTML) > 0 Then
Begin
Category1 := 'No Connections';
Category3 := '';
AddCustomFieldValueByName('Connections', Category1);
AddCustomFieldValueByName('Connect', Category3);
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - No Connections found for this title');
End Else Begin
// Extract the relevant section for categories
If PosFrom('<span class="ipc-simple-select__container" data-testid="jumpTo">', HTML, curPos) > 0 Then Begin
curPos := PosFrom('<span class="ipc-simple-select__container" data-testid="jumpTo">', HTML, curPos);
endPos := PosFrom('</select></span></span>', HTML, curPos);
Category2 := Copy(HTML, curPos, endPos - curPos);
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Extracted Category2 section: ' + Category2);

// Parse the options and category names
curPos := 1;
Category1 := '';
While PosFrom('<option', Category2, curPos) > 0 Do
Begin
curPos := PosFrom('<option', Category2, curPos) + Length('<option');
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - curPos at option tag: ' + IntToStr(curPos));

optionValue := TextBetween(Category2, 'value="', '">', False, curPos);
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Extracted optionValue: ' + optionValue);

categoryName := TextBetween(Category2, '">', '</option>', False, curPos);
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Extracted categoryName: ' + categoryName);

// Format the category link
//Category1 := Category1 + '<link url="' + MovieURL + optionValue + '">' + categoryName + '   •   </link>';
// Category1 := Category1 + '<link url="' + MovieURL + optionValue + '">' + categoryName + '</link>   •   ';
Category1 := Category1 + '<link url="' + MovieURL + optionValue + '">' + categoryName + '</link>    ';
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Formatted and added category link: ' + '<link url="' + MovieURL + optionValue + '">' + categoryName + '   •   </link>');

// Move to the next position
curPos := PosFrom('</option>', Category2, curPos) + Length('</option>');
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Moved curPos to next option tag: ' + IntToStr(curPos));
End;

// Remove the trailing "   •   " if it exists and keep the last </link> intact
If Copy(Category1, Length(Category1) - 13, 7) = '   •   ' Then
Begin
Category1 := Copy(Category1, 1, Length(Category1) - 14) + '</link>';
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Removed trailing "   •   " and kept last </link> in Category1');
End;

LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Final Category1 before combining: ' + Category1);

// Combine and format the final result
//Category1 := '<link url="' + MovieURL + '">Connections:    </link>' + Category1;
if Category1 <> '' then
Category1 := '<link url="' + MovieURL + '">Connections:</link>     •     ' + Category1
Else
Category1 := '<link url="' + MovieURL + '">Connections:</link>          ' + Category1;
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Combined and formatted final Category1: ' + Category1);

// Store the result in the custom field
AddCustomFieldValueByName('Connections', Category1);
LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Stored result in (CF~Connections~)');

LogMessage('Function ParsePage_IMDBMovieCONNECTIONS - Final Category1 stored: ' + Category1);
LogMessage('===Function ParsePage_IMDBMovieCONNECTIONS - Finished (CF~Connections~) Parsing ===');
End;

I tested it, IMDB_Movie_[EN][Selenium]-v4.psf and IMDB_People_[EN][Selenium]-v4.psf work fine, with some errors in certain information, where it will be necessary to fix parts of the code in the scripts. The errors have been there for a long time and have not been fixed. The code fixes will be described and added below.

As for FilmAffinity_Movie_[EN][Selenium]-v4.psf, I have not tested it.

I have completed FullCredits function, so now all the data can be imported again to PVD. Before I publish it I want to check 2 things:
1. What to do with Reference page code, and should I exclude its options from Script Configurator, since now it is completely not needed.
2. To check People script and if I can fix it quickly, then I will publish all the scripts and files again in one, final package for this IMDb html layout change.

After that please check the scripts and let me know what doesn't work

First of all, Function ParsePage_IMDBMovieREFERENCE should be moved to the end of the script before Function ParsePage.

Secondly, the entire part of the reference page code should be left and, as mentioned above, before Function ParsePage, or even better, it should be moved to the very end of the script, where there is already a History of changes, so that it can be re-included in the script if necessary in the future or completely blocked. All its options should also be excluded from both the script code and the script configurator, because now, as mentioned, it is no longer needed.

As for the People script, I don't know what works or doesn't work, because I don't use it.

Oh, sorry to hear. I even didn't know it worked earlier and how it could work since I never published geckodriver version??? Probably I don't understand the context of your message.

I changed all parts of the code where there is a Chrome and Chromedriver options record to Firefox and Geckodriver options record and the Selenium_Chrome_Base_page_v4.py script also worked using the Firefox browser.

As I mentioned before, with the latest Selenium_Chrome_Base_page_v4.py script update and the Chrome and Chromedriver options record, I changed it to Firefox and Geckodriver options record, but the Selenium_Chrome_Base_page_v4.py script no longer works at all using the Firefox browser.

We already talked about this a while ago in another PVD Python Scripts topic.

I have optimized Selenium_Chrome_Base_page_v4.py script so now downloading of the IMDb main page should be dramatically faster - around 18 seconds on my computer.

The Selenium_Chrome_Base_page_v4.py script works fine. However, it doesn't work at all with Firefox and Geckodriver options.

Changed the path for Geckodriver in Selenium_Chrome_Base_page_v4.py for Firefox script in 7.z file.

You need the latest chromedriver.exe, otherwise it won't work.

And this tip.

For the Episode List page, it would be necessary and even better if a script was made specifically for it.