Personal Video Database

English => Support => Topic started by: VVV_Easy_Programing on June 16, 2018, 09:31:27 am

Title: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on June 16, 2018, 09:31:27 am
The last oficial version has been the 1.0.2.7 published in the 05 November 2013 by Nostra, the oficial programer and owner of the program.
After this date, the information providers has changed his pages and, the more important, they have passed to a secure protocol HTTPS:  PVD is not prepared for this protocol and crashes with a “IO Handler Value is not Valid” error.

Ivek23, the person that have supported the forum and the scripts programs with a important personal effort, thank you very much, has surrounded this problem installing and configuring a proxy, Proxomitron, but this require some computer skills and PVD is no longer portable.

So, we have made a MOD version of PVD 1.0.2.7 with the next characteristics:
To surround the HTTPS problem, it has been necessary to make an intermediate download in the file of the pages. This makes HTTPS scripts slower than with the Proxomitron solution, you must choose which solution to use because they are not compatible (but the working scripts can be easily adapted for exchange).

Until Nostra awakens from his Odinsleep, we pray to God that it will be soon, this is our humble contribution to this superb program, the most flexible and powerful movie data base:

http://vvveasy.altervista.org/wp-content/uploads/2019/03/PVD_1.0.2.7_MOD-V20190308.zip (http://vvveasy.altervista.org/wp-content/uploads/2019/03/PVD_1.0.2.7_MOD-V20190308.zip)

(The author strongly recommends using the SyMenu (https://www.ugmfree.it/) free launcher to install and keep updated the portable Personal Video Database 1.0.2.7 MOD version. It's easy and completely free without charges, advertisements, etc. (same philosophy as PVD) and perhaps you discover other advantages (for instance the Win10 users don't need use the infamous Windows Tiles))

Note: By Ivek23 request, below is included the adaptation of version 0.9.9.21.  It does not have the same guarantees as 1.0.2.7 especially because this version of PVD has a different scripting engine and the People part is less developed-DISCONTINUED:PVD_0.9.9.21_MOD-V20180922.zip (http://vvveasy.altervista.org/wp-content/uploads/2018/09/PVD_0.9.9.21_MOD-V20180922.zip)


Temporal Acumulative Scripts Patch date: Not needed


You can also find more information about Custom Fields in the topic Possibles improving Custom Fields working in MOD version. (http://www.videodb.info/forum_en/index.php/topic,4141.msg20789.html#msg20789)
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on June 16, 2018, 09:31:52 am
Usage:
Important advice: TheMovieDB is the recommended script, for speed and reliability. Take notice, that your are downloading full pages of internet with images, ad's, etc. For instance if you retrieved all IMDB info, your are downloading 1+5 pages for each movie.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on June 16, 2018, 09:32:47 am
MOD version Update:
If you use MOD version and you wants update to the newer MOD versions by yourself (advanced users) you could do the following steps (let us know your problems to be able to improve the guide):
(Remember: For normal users, the author strongly recommends using the SyMenu (https://www.ugmfree.it/) free launcher to install and keep updated the portable Personal Video Database 1.0.2.7 MOD version)


Moving from original PVD version to MOD version.
Perhaps, there may be many PVD users who have a movie collection in a PVD original installation and don't want to start over. For them, we suggest the following steps.

"Slow Copy-Paste" own PVD installation/collection adaptation for normal 1.0.2.7 users (not work with 0.9.9.21):

"Dirty" own PVD installation/collection adaptation for advanced users:

Controling the retreived information in PVD MOD:
There are two way to control the saved of information in the PVD database.
a) The PVD standard with the 'Set Overwrite Options for the selected plugin' (the "Calendar" button, at right of the script 'Import name').
b) The custom fields and others options aren't included in this menu so the scripts have hardcode parameters to managed it.
Now in PVD MOD can be easily changed launching the first script '**** Scripts Configuration ****'. These are its powerful capabilities and how use it:
•) Launch the PVD MOD Configuration Script (you must have selected one movie or one person):

(http://www.videodb.info/forum_en/index.php?action=dlattach;topic=4134.0;attach=6704)

•) Then the PDV MOD Script Configuration windows opens:

(http://www.videodb.info/forum_en/index.php?action=dlattach;topic=4134.0;attach=6708)

1) Choose the script configuration sheet (Orange arrow)
2) You can activate or deactivate a script checking the Active box (Blue oval). The activate scripts are showed in PVD.
3) Check or un-check the desired option (Green oval). In the example you can avoid download all the IMDB information pages so the script will go faster. This is interesting when you are updating only some particular info, for instance the IMDBRating, than you don't need the actors.
4) There is other numeric parameters that you can change, for instance, the number of actors saved in PVD.
5) Finaly you must saved it (vertical Red arrow). PVD MOD restarts in the same position because it needs compile the scripts with the new parameters.

The author recomends you strongly use **** Scripts Configuration ****. You can gain a lot of time.

Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on June 16, 2018, 09:39:03 am
Changelog:
Last version always in the first post
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on June 16, 2018, 11:39:26 am
Other solutions and scripts will be ready for the near future, according to the users' wishes. You can also visit the VVV_Easy Blog web site of our VVV_Easy_Programing user at the link below.

http://vvveasy.altervista.org/personal-video-database/?doing_wp_cron=1525497049.1403949260711669921875
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on June 16, 2018, 11:49:33 am
In the field NEWS I gave this record to all users of our forum as a notice.
Quote
Personal Video Database 1.0.2.7 MOD is now public available:
http://vvveasy.altervista.org/wp-content/uploads/2018/11/PVD_1.0.2.7_MOD-V20181101.zip

No more.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: meriator on June 22, 2018, 01:20:55 pm
thank you   :)
I will try it out as soon as possible
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Am-rA on June 30, 2018, 08:38:47 pm
 :o
thanks my friends, i´m  impressed, in a moment i install it
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: clandher on July 08, 2018, 11:52:21 am
Hi

This is a very good job, but I have a simple question. How could I made a simple search using a word to know if there is or not in the database (titles) ? I couldn't find this option.

Thanks
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on July 10, 2018, 09:54:00 am
Hi

This is a very good job, but I have a simple question. How could I made a simple search using a word to know if there is or not in the database (titles) ? I couldn't find this option.

Thanks

Search tab, then proceed as shown in the picture [Title and to write the title of the movie you want to check in the empty box  (see sshot-52)] for an example of a Robin Hood title search. This should give you a result if a particular address is already in your database.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: clandher on July 11, 2018, 01:56:18 am
Thanks and sorry. It was my mistake. Some how I downloaded and install PVD_0.9.9.21_MOD-V.2018.06.16
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: jondak on July 17, 2018, 01:30:35 pm
PVD_0.9.9.21_MOD-V.2018.06.16

When opening the database i had used till the mod in 0.9.9.21 it gave the error that is found in atachement..

I just replaced fbembed.dll with the one from the default install that is version 2.51.26351 and it open the database without problems.

I don't remember if I updated the file manually or was updated when i installed firebird server or when i tested 1.0.27 version.

So in case someone has a similar problem this can help.


Question:

its there a way to rename the IMDB_Votes custom field to IMDB Votes that i had from the original script?

Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on July 17, 2018, 02:17:45 pm
PVD_0.9.9.21_MOD-V.2018.06.16

Question:

its there a way to rename the IMDB_Votes custom field to IMDB Votes that i had from the original script?

Yes, of course, open the script, find the '' // Get ~ imdbrating ~, ~ IMDB_Votes ~ '' code section, and add or change [(depending on you) - see the added quote code section]. You can then use the original IMDB Votes custom box.

Quote
Function ParsePage_IMDBMovieBASE(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If not results
    //Retrieve: ~title~, ~year~, ~origtitle~, ~poster~ / ~imdbrating~, ~IMDB_Votes~ (Custom Field) / ~TOP_250~(Custom Field) /
    //          If Not(GET_FULL_CREDIT): ~crew~ctDirectors,ctWriters,ctComposers,ctProducers(Not in base page), ctActors
    //         ~description~ / ~category~ "keywords" / ~tagline~ / ~genre~
    //         If Not(GET_FULL_MPAA) ~mpaa~
    //         ~country~ / ~rdate~ in contry provider local IP geolocation
    //         If Not(GET_FULL_AKA) ~aka~.
    //         ~budget~ / ~money~ / ~studio~ "Production Co"
    //         If GET_FEATURES ~features~
  Var
      curPos,endPos,index:Integer;
      ItemValue,ItemList,ImageFile:String;
      titleValue:String;
      Name,Role,PersonURL:String;
  Begin
    LogMessage('Function ParsePage_IMDBMovieBASE BEGIN======================|');
    //Because the script don't retreives the data in the page order a token search for the first curPos position or block select is mandatory
    //Get ~title~, ~year~, ~origtitle~, ~poster~
    //Get all "raw" title summary (in raw because we need the hidden links, we avoid "complete" token in strings which opens/closes)
    ItemList:=TextBetWeenFirst(HTML,'<div class="title_wrapper"','div class="plot_summary_wrapper">'); //WEB_SPECIFIC.
    If (Length(ItemList)>0) Then Begin           
        titleValue:=TextBetWeenFirst(ItemList,'<h1 itemprop="name" class="">','<');       //Strings which opens/closes the data. WEB_SPECIFIC
        AddFieldValueXML('title',titleValue);
        LogMessage('      Get result title:'+titleValue+'||');
        ItemValue:=TextBetWeenFirst(ItemList,'(',')');                                 //Strings which opens/closes the data. WEB_SPECIFIC
        AddFieldValueXML('year',ItemValue);
        LogMessage('      Get result year:'+ItemValue+'||');
        ItemValue:=TextBetWeenFirst(ItemList,'<div class="originalTitle">','<');          //Strings which opens/closes the data. WEB_SPECIFIC
        If (Length(ItemValue)=0) Then ItemValue:=titleValue;                           //Provider hides the original title if same that title. WEB_SPECIFIC
        AddFieldValueXML('origtitle',ItemValue);
        LogMessage('      Get result origtitle:'+ItemValue+'||');
        ItemValue:=TextBetWeenFirst(ItemList,BASE_URL_IMAGE_PRE_TRUE,'.');                 //Get poster code. Strings which opens/closes the data. WEB_SPECIFIC
        If (Length(ItemValue)>0) then begin
            ItemValue:=BASE_URL_IMAGE_PRE_TRUE + ItemValue;                             //Base poster URL without '.jpg'. WEB_SPECIFIC
            ImageFile:=GetAppPath+'Scripts\'+BASE_DOWNLOAD_FILE_IMAGE_NAME+'-Poster.jpg'
            // Avoid HTTPS redirection: Download https image to file  OutPutFile=
            If (1=DownloadImage(ItemValue + '._V1_UY' + IntToStr(MAX_IMAGE_HEIGTH) + '_.jpg',ImageFile)) then begin  //Dowload in the selected user max size. WEB_SPECIFIC
                AddImageURL(itPoster,ImageFile);    //Get the photo from a file even if the next line we log the true URL.
                LogMessage('      Get result poster:'+ItemValue + '._V1_UY' + IntToStr(MAX_IMAGE_HEIGTH) + '_.jpg'+'||');
            end else if (1=DownloadImage(ItemValue +'.jpg',ImageFile)) then begin  //Dowload in the web base size. WEB_SPECIFIC
                AddImageURL(itPoster,ImageFile);    //Get the photo from a file even if the next line we log the true URL.
                LogMessage('      Get result poster:' + ItemValue + '.jpg'+'||');
            end;       
        end;
    End;
    //Get ~imdbrating~, ~IMDB_Votes~
    curPos:=Pos('<span itemprop="ratingValue">',HTML);                      //WEB_SPECIFIC.
    If 0<curPos Then Begin                         
       ItemValue:=TextBetWeen(HTML,'<span itemprop="ratingValue">','<',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=StringReplace(ItemValue,',','.',True,True,False);                           //Decimal comma spanish separator to point english separator.
       AddFieldValueXML('imdbrating',ItemValue);
       LogMessage('      Get result imdbrating:'+ItemValue+'||');
       ItemValue:=TextBetWeen(HTML,'<span class="small" itemprop="ratingCount">','</span>',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=StringReplace(ItemValue,'.','',True,True,False);            //Remove thousand point spanish separator.
       ItemValue:=StringReplace(ItemValue,',','',True,True,False);            //Remove thousand comma english separator.
       AddCustomFieldValueByName('IMDB_Votes',ItemValue);
       AddCustomFieldValueByName('IMDB Votes',ItemValue);
       LogMessage('      Get result IMDB_Votes:'+ItemValue+'||');
    End;

Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: jondak on July 17, 2018, 05:40:13 pm
Thank you Ivek23. Modified and works well.

Found a minor bug:

Movie:  Ratatouille (2007) https://www.imdb.com/title/tt0382932/

MPAA retrieved: Rated PG for mild action                                                                        Edit     

It adds the "Edit" at the end of the MPAA rating retrived. Other movies works well. Will continue testing.

Cheers.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on July 18, 2018, 06:00:01 am
Found a minor bug:

Movie:  Ratatouille (2007) https://www.imdb.com/title/tt0382932/

MPAA retrieved: Rated PG for mild action 

A minor bug will be corrected at the next update of the script.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on August 01, 2018, 08:54:49 am
Thank you Ivek23. Modified and works well.
Found a minor bug:
Movie:  Ratatouille (2007) https://www.imdb.com/title/tt0382932/
MPAA retrieved: Rated PG for mild action                                                                        Edit     
It adds the "Edit" at the end of the MPAA rating retrived. Other movies works well. Will continue testing.
Cheers.

When you post this bug I read the IMDB page and I saw that there was a commentary in the Parents Guide that said "Proposed MPAA: Rated PG for mild action". I think that this caused the confusion of the script. Today this comment has disappeared and I am not able to reproduce the error for correct it.
Thank you very much for your alert.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on August 01, 2018, 12:15:25 pm
PVD_0.9.9.21_MOD-V.2018.06.16
When opening the database i had used till the mod in 0.9.9.21 it gave the error that is found in atachement..
I just replaced fbembed.dll with the one from the default install that is version 2.51.26351 and it open the database without problems.
I don't remember if I updated the file manually or was updated when i installed firebird server or when i tested 1.0.27 version.
So in case someone has a similar problem this can help.
Question:
its there a way to rename the IMDB_Votes custom field to IMDB Votes that i had from the original script?

You give me a idea: In the next MOD version I'll tried to update all possible dll. Please alert me if with this change appears news bugs.

BTW, I advise you to keep the names of the fields instead of changing the scripts: It’s to work once but it remains forever.
If you want to preserve your DB, you can do it without data loose in Preferences / Movies / Custom Items, select the field, push the “Rename” button and write the new standard MOD name.
The use of spaces in the name of fields is not a very good idea. I changed it many years ago, I think because it gave me problems with the Export Templates.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on August 01, 2018, 01:41:27 pm
Changelog:
Last version always in the first post
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on August 03, 2018, 06:49:59 am
Changelog:
  • MOD V.2018.08.01; VVV: DLL renovation (Firebird and SQLite).
                                           Change values of CHEAT-PREFIX-URL (better server the ancient sometimes fails in Win10) and DUMMY-HTML-FILE (always Portable.bat).
                                           IMDB_People_[EN][HTTPS] Awards retrieve developed.
                                           Other little improvements.
Last version always in the first post

An error in IMDB_ [EN] [HTTPS] for the transfer of awards where the year of the prize is missing.

Before
Quote
AddAward(EventName, AwardName, AwardCategory, AwardRecipient, EventYear, AwardWon);

Now
Quote
AddAward(EventName, AwardName, AwardCategory, AwardRecipient, '', AwardWon);
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on August 03, 2018, 07:44:24 am
Really this is a correction of a first version error. The Script Manual says:

    procedure AddAward(Event, Award, Category, Recipient, Year : String; const Won : Boolean);
    Adds an award to a record. Parameters:
    Event = name of the event
    Award = award name
    Category = award category
    Recipient = for movie records this variable should contain the name of a specific person who won the award and
                     for people records the variable should contain the title of a movie for which the person won the award
    Year = release year of a movie (only applicable when adding award to a person record)
    Won = set to true if the recipient won the award and to false otherwise


In the first version the movie script added the "EventYear" (normally is the next year of the movie). But now we can add awards with the people script and then the year must be the movie year so we could have the same award with different years. So, in order to respect the Script Manual, the movie scripts doesn't write the year and is the people scripts who writes the year.
BTW, I have hesitated to write the year of the movie instead of leaving it blank but I do not know the consistency of the data. What do you think?
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on August 04, 2018, 07:08:04 am
Really this is a correction of a first version error. The Script Manual says:

    procedure AddAward(Event, Award, Category, Recipient, Year : String; const Won : Boolean);
    Adds an award to a record. Parameters:
    Event = name of the event
    Award = award name
    Category = award category
    Recipient = for movie records this variable should contain the name of a specific person who won the award and
                     for people records the variable should contain the title of a movie for which the person won the award
    Year = release year of a movie (only applicable when adding award to a person record)
    Won = set to true if the recipient won the award and to false otherwise

This needs to be read on several occasions, but it is not yet certain that it will be clarified correctly as it was in your case.

    procedure AddAward(Event, Award, Category, Recipient, Year : String; const Won : Boolean);
    Adds an award to a record. Parameters:
.
.
.
    Recipient = for movie records this variable should contain the name of a specific person who won the award and
                     for people records the variable should contain the title of a movie for which the person won the award
    Year = release year of a movie (only applicable when adding award to a person record)
...

He also writes that movie awards have a year of prize and the name of the actor he has received.

He also writes for people awards, where the actor won the prize for a movie without recording for release year.

In the first version the movie script added the "EventYear" (normally is the next year of the movie). But now we can add awards with the people script and then the year must be the movie year so we could have the same award with different years. So, in order to respect the Script Manual, the movie scripts doesn't write the year and is the people scripts who writes the year.

In the first version, the movie script was all prailono for aswards as it should be.

BTW, I have hesitated to write the year of the movie instead of leaving it blank but I do not know the consistency of the data. What do you think?

The correct thing is that movie awards code is the same as it was in the first version of the movie script.

It is right for people awards to record movie names not to include a record for release year. I know from my own experience in testing my people awards script, where my record looks like this [Casablanca (1942)]. The correct record is only Casablanca.

In the first version the movie script added the "EventYear" (normally is the next year of the movie).

This is not always true. A good example is Casablanca (1942) (http://www.imdb.com/title/tt0034583/) and the Academy Awards, USA (https://www.imdb.com/title/tt0034583/awards?ref_=tt_awd) , where awards were only awarded in 1944.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on August 06, 2018, 10:55:59 am
Well, I don't understand all of your arguments but I "buy" : Return to the first solution for the movie
Quote
AddAward(EventName, AwardName, AwardCategory, AwardRecipient, EventYear, AwardWon);

and the same use EventYear for the People script in order to warrant the same data (I have done the test with Casablanca and Michael Curtiz).

Ivek, thank you very much for your explanation (I attach the scripts and the screenshots).

BTW, I want to publish a new release the next Wednesday, 8/8/2018 (nice date, isn't it?) with those scripts, proxy support and other bugs corrected.

Dear users, please, tell me all you bug discovers in order to correct it fast for this next version (and the 9/8/2018 we'll can go in holidays  ;D )
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on August 07, 2018, 06:54:56 am
IMDB_People_[EN][HTTPS]

Now that awards have been added, IMDB_People_ [EN] [HTTPS] script does not download (adds to the database) more photos of the players. It was not previously (it was a download) until the award code was added to the script. Obviously, I tested a script with a award code (photos did not add in the database) and script, when I removed the award code (photos added to the database), that the award code prevents the addition of photos to the database.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on August 08, 2018, 08:18:10 am
Is the famous problem with downloading images in the people PVD part. I have forgotten to keep the rondabout result througth the awards part. It will be solved easily.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on August 08, 2018, 10:31:31 am
Changelog:
Last version always in the first post
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: jondak on August 13, 2018, 09:53:29 am
Script Movie Configuration bug for http://vvveasy.altervista.org/wp-content/uploads/2018/08/PVD_0.9.9.21_MOD-V.2018.08.08.zip




Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on August 13, 2018, 05:35:30 pm
I think that what is the problem: can you edit the PVdBScriptConfig.ini with notepad and change GET_FUL_FEATURES by GET_FULL_FEATURES with two L and try again ?
Thank you for your help.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: jondak on August 14, 2018, 08:06:03 am
Yes, with that modification it worked.

Thank you.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on August 14, 2018, 07:54:48 pm
 :) The problem of the hurry. It will be corrected in the next version. Thank you.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on August 19, 2018, 02:20:07 pm
An many errors in IMDB_ [EN] [HTTPS] script.

Errors are in Rating, Top 250 IMDB Votes, Studio and Full Actors in the Full Cast and Crew section. This happened because of the change of source code on IMDB Movies web pages. I've already tested the code corrections and code corrections will be published in a few days. I beg you for patience, because I'm a little more close to my free time.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: sihyp on August 19, 2018, 02:45:18 pm
Yes, you've been working here so hard.
Yvek, you deserve it. :)
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: AimHere on August 20, 2018, 05:36:32 pm
Other solutions and scripts will be ready for the near future, according to the users' wishes.

I'd like to request that the AdultDVDEmpire script be adapted for this mod. I've been sort of maintaining the ADE script in the past, but I can lo longer get PVD and Proxomitron to play nice together. From what I can glean from looking at the mod's scripting, the mod uses an external program to download all https:// web pages, then parses them locally. This is beyond my capabilities to adapt.

Thanks,
aimhere
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on August 22, 2018, 06:26:13 am
Yes, you've been working here so hard.
Yvek, you deserve it. :)

Thanks, I'm doing my best.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 02, 2018, 07:58:35 am
IMDB_ [EN] [HTTPS] script

In the tests I found several errors that I corrected and added some of the code parts for better information transfer. Added some new code sections for downloading additional information.

An many errors in IMDB_ [EN] [HTTPS] script.

Errors are in Rating, Top 250 IMDB Votes, Studio and Full Actors in the Full Cast and Crew section. This happened because of the change of source code on IMDB Movies web pages. I've already tested the code corrections and code corrections will be published in a few days. I beg you for patience, because I'm a little more close to my free time.

First of all, what I've corrected for changing the source code on IMDB pages.

Full Cast and Crew section

I've repaired pieces of code for transmitting actors information.

Quote
Function ParsePage_IMDBMovieCREDIT(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit;
    //Retrieve: ~crew~ctDirectors,ctWriters,ctComposers,ctProducers
    //          ~actors~ ctActors
  Var
    curPos,endPos,index:Integer;
    ItemList:String;
    Name,Role,PersonURL:String;
  Begin
    LogMessage('Function ParsePage_IMDBMovieCREDIT BEGIN=====================||');
    Result:=prFinished;  //It will change to prError if any big problem with exit;
  //Get ~crew~ctDirectors,ctWriters,ctComposers,ctProducers
.
.
    //Get  to "Cast" ~actors~ ctActors
    //Go Cast list
    curPos:=Pos('<table class="cast_list">',HTML);                                     //Strings start which opens the block content data. WEB_SPECIFIC
    curPos:=curPos+Length('<table class="cast_list">');                                //Strings end which opens the block content data.  WEB_SPECIFIC
    //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~
    endPos:=PosFrom('</div>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
    ItemList:=Copy(HTML,curPos,endPos-curPos);
    //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');
    curPos:=Pos('<td class="primary_photo">',ItemList)                               //String which opens the subList data. WEB_SPECIFIC
    index:=1;
    While curPos>0 Do Begin
        If (index>PEOPLE_LIMIT) Then break;     //Limited depassed (Remember index begin in 0).
        //Get PersonURL (Always must it has)
        PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF; //Strings which opens/closes the data. WEB_SPECIFIC
        LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
        //Get Name (Always must it has)
        //Name:=TextBetWeen(ItemList,'itemprop="name">','<',false,curPos);                             //Strings which opens/closes the data. WEB_SPECIFIC
        Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                             //Strings which opens/closes the data. WEB_SPECIFIC      
        LogMessage('      Parse Results Name:'+Name+'||');
        //Get Role
        //Role:=TextBetWeen(ItemList,'<td class="character">','</a>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC
        Role:=TextBetWeen(ItemList,'<td class="character">','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC         
        LogMessage('      Parse Results Role:'+Role+'||');
      Role:=StringReplace(Role,'       (',' (',True,False,True);
      Role:=StringReplace(Role,'(uncredited)                    ','(uncredited) • ',True,False,True);
      Role:=StringReplace(Role,')                    ',') ',True,False,True);
      Role:=StringReplace(Role,' /              ',' - ',True,False,True);
      Role:=StringReplace(Role,'         / ...                    ',' • ',True,False,True);
      Role:=StringReplace(Role,'                  ',' • ',True,False,True);      
      Role:=StringReplace(Role,'      (uncredited)',' (uncredited)',True,False,True);
      LogMessage('      Parse Results Role_1:'+Role+'||');   
   
        AddMoviePerson(Name,'',Role,PersonURL,ctActors);
        LogMessage('      Get results cast:#'+IntToStr(index)+'|'+Name+'|'+Role+'|'+PersonURL+'||ctActors');                                               
        curPos:=PosFrom('<td class="primary_photo">',ItemList,curPos);                                  //String which opens the Web Result item List data. WEB_SPECIFIC
        index:=index+1;
    End;
    LogMessage('Function ParsePage_IMDBMovieCREDIT END=====================||');
  End; //BlockClose

Note:
At Actors is also part of the role code for the actors series informations.


I corrected the pieces of code for transmitting information to the director, writer, producer and music, because in certain cases it transmitted wrong information with Full Cast and Crew pages. This happened when there was no information for actors, writer, producer or music .

For example, the same name was added to the database for the director and producer, even though the producer on the Full Cast and Crew site was not mentioned. Director, writer, producer and music do not have role information in dastabase. For the correct transfer of information in the database, I added all the roles to everyone. Role and LogMessage are also for the debug log file. The role is not visible in the database information unless the transfer was in the custom field.

Part 1:
Quote
  //Get ~crew~ctDirectors,ctWriters,ctComposers,ctProducers
    //Go to "Directed by" ~crew~ctDirectors
    //curPos:=Pos('Directed by',HTML);                                                   //Strings start which opens the block content data. WEB_SPECIFIC
    curPos:=Pos('<h4 class="dataHeaderWithBorder">Directed by',HTML);                                                   //Strings start which opens the block content data. WEB_SPECIFIC   
    curPos:=PosFrom('</h4>',HTML,curPos);                                              //Strings end which opens the block content data.  WEB_SPECIFIC
    curPos:=curPos+Length('</h4>');                                                    //Strings end which opens the block content data.  WEB_SPECIFIC
    //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~ n the ~crew~
    endPos:=PosFrom('</table>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
    ItemList:=Copy(HTML,curPos,endPos-curPos);
    //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');
    curPos:=Pos('<td class="name">',ItemList)                                         //String which opens the subList data. WEB_SPECIFIC
    index:=1;
    While curPos>0 Do Begin
        If (index>PEOPLE_LIMIT) Then break;     //Limited depassed (Remember index begin in 0).
        //Get PersonURL (Always must it has)
        PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF;     //Strings which opens/closes the data. WEB_SPECIFIC
        LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
        //Get Name (Always must it has)
        //Name:=TextBetWeen(ItemList,'>','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC
        Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC      
        LogMessage('      Parse Results Name:'+Name+'||');
      //Get Role NO: PVD don't save Role in crew people
      //Get Role
      Role:=TextBetWeen(ItemList,'<td class="credit">','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC
      LogMessage('      Parse Results Role:'+Role+'||');

        AddMoviePerson(Name,'','',PersonURL,ctDirectors);
        LogMessage('      Get results Director:#'+IntToStr(index)+'|'+Name+'|'+PersonURL+'||ctDirectors'); 
        curPos:=PosFrom('<td class="name">',ItemList,curPos)                                            //String which opens the subList data. WEB_SPECIFIC
        index:=index+1;
    End;           
    //Go to "Writer:" or "Writers:" ~crew~ctWriters
   curPos:=Pos('<h4 class="dataHeaderWithBorder">Writing Credits',HTML);
   If 0<curPos Then Begin   

    //curPos:=Pos('Writing',HTML);                                                       //Strings start which opens the block content data. WEB_SPECIFIC
      curPos:=Pos('<h4 class="dataHeaderWithBorder">Writing Credits',HTML);                                                       //Strings start which opens the block content data. WEB_SPECIFIC   
      curPos:=PosFrom('</h4>',HTML,curPos);                                              //Strings end which opens the block content data.  WEB_SPECIFIC
      curPos:=curPos+Length('</h4>');                                                    //Strings end which opens the block content data.  WEB_SPECIFIC
      //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~
      endPos:=PosFrom('</table>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
      ItemList:=Copy(HTML,curPos,endPos-curPos);
      //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');
      curPos:=Pos('<td class="name">',ItemList)                                            //String which opens the subList data. WEB_SPECIFIC
      index:=1;
      While curPos>0 Do Begin
         If (index>PEOPLE_LIMIT) Then break;     //Limited depassed (Remember index begin in 0).
         //Get PersonURL (Always must it has)
         PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF;     //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
         //Get Name (Always must it has)
         //Name:=TextBetWeen(ItemList,'>','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC
         Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results Name:'+Name+'||');
         //Get Role NO: PVD don't save Role in crew people
         //Get Role
         Role:=TextBetWeen(ItemList,'<td class="credit">','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results Role:'+Role+'||');   
      
         AddMoviePerson(Name,'','',PersonURL,ctWriters);
         LogMessage('      Get results Writer:#'+IntToStr(index)+'|'+Name+'|'+PersonURL+'||ctWriters'); 
         curPos:=PosFrom('<td class="name">',ItemList,curPos)     //String which opens the subList data. WEB_SPECIFIC
         index:=index+1;
      End;
    End;
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 02, 2018, 08:01:34 am
Part 2:

Quote
    //Go to "Producers" ~crew~ctProducers
   curPos:=Pos('<h4 class="dataHeaderWithBorder">Produced by',HTML);
   If 0<curPos Then Begin
      curPos:=Pos('<h4 class="dataHeaderWithBorder">Produced by',HTML);                                                         //Strings start which opens the block content data. WEB_SPECIFIC   

      //curPos:=Pos('Produced',HTML);                                                         //Strings start which opens the block content data. WEB_SPECIFIC
      curPos:=PosFrom('</h4>',HTML,curPos);                                              //Strings end which opens the block content data.  WEB_SPECIFIC
      curPos:=curPos+Length('</h4>');                                                    //Strings end which opens the block content data.  WEB_SPECIFIC
      //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~
      endPos:=PosFrom('</table>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
      ItemList:=Copy(HTML,curPos,endPos-curPos);
      //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');;
      curPos:=Pos('<td class="name">',ItemList)                                               //String which opens the subList data. WEB_SPECIFIC
      index:=1;
      While curPos>0 Do Begin
         If (index>PEOPLE_LIMIT) Then break;     //Limited depassed (Remember index begin in 0).
         //Get PersonURL (Always must it has)
         PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF;     //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
         //Get Name (Always must it has)
         //Name:=TextBetWeen(ItemList,'>','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC
         Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results Name:'+Name+'||');
         //Get Role NO: PVD don't save Role in crew people
         //Get Role
         Role:=TextBetWeen(ItemList,'<td class="credit">','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results Role:'+Role+'||');

         AddMoviePerson(Name,'','',PersonURL,ctProducers);
         LogMessage('      Get results Producers:#'+IntToStr(index)+'|'+Name+'|'+PersonURL+'||ctProducers'); 
         curPos:=PosFrom('<td class="name">',ItemList,curPos)                                                   //String which opens the subList data. WEB_SPECIFIC
         index:=index+1;
      End;   
    End;
    //Go to "Music" ~crew~ctComposers
   curPos:=Pos('<h4 class="dataHeaderWithBorder">Music by',HTML);
   If 0<curPos Then Begin

      //curPos:=Pos('Music',HTML);                                                         //Strings start which opens the block content data. WEB_SPECIFIC
      curPos:=Pos('<h4 class="dataHeaderWithBorder">Music by',HTML);                                                         //Strings start which opens the block content data. WEB_SPECIFIC      
      curPos:=PosFrom('</h4>',HTML,curPos);                                              //Strings end which opens the block content data.  WEB_SPECIFIC
      curPos:=curPos+Length('</h4>');                                                    //Strings end which opens the block content data.  WEB_SPECIFIC
      //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~
      endPos:=PosFrom('</table>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
      ItemList:=Copy(HTML,curPos,endPos-curPos);
      //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');List+'||');
      curPos:=Pos('<td class="name">',ItemList)                                               //String which opens the subList data. WEB_SPECIFIC
      index:=1;
      While curPos>0 Do Begin
         If (index>PEOPLE_LIMIT) Then break;     //Limited depassed (Remember index begin in 0).
         //Get PersonURL (Always must it has)
         PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF;     //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
         //Get Name (Always must it has)
         //Name:=TextBetWeen(ItemList,'>','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC
         Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC         
         LogMessage('      Parse Results Name:'+Name+'||');
         //Get Role NO: PVD don't save Role in crew people
         //Get Role
         Role:=TextBetWeen(ItemList,'<td class="credit">','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results Role:'+Role+'||');

         AddMoviePerson(Name,'',Role,PersonURL,ctComposers);
         LogMessage('      Get results Composers:#'+IntToStr(index)+'|'+Name+'|'+PersonURL+'||ctComposers'); 
         curPos:=PosFrom('<td class="name">',ItemList,curPos)                                                   //String which opens the subList data. WEB_SPECIFIC
         index:=index+1;
      End;
    End;
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 02, 2018, 08:34:22 am
IMDB_ [EN] [HTTPS] script

ParsePage_IMDBSearchTitle

I corrected the pieces of code of search results for movies, where there are now missing posters.

Example for this link:
https://www.imdb.com/search/title?title=Lawrence+Of+Arabia&title_type=feature&release_date=,&view=simple
Poster link:
https://m.media-amazon.com/images/G/01/imdb/images/nopicture/140x209/film-4001654135._CB470041848_.png


Quote
Function ParsePage_IMDBSearchTitle(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prDownload; (Global var DownloadURL=IMDB page) if there is one results.
    //     Result:=prList; if there are several results. (Don't work with Preferences/Plugings/Silent Mode).
    //     Result:=prError; If not results
  Var
      curPos,ResultsNumber:Integer;
      ItemValue,ImageFile:String;
      Title,Year,MovieURL,PreviewURL:String;
  Begin
    LogMessage('Function ParsePage_IMDBSearchTitle BEGIN======================|');
    LogMessage('Global Var-Mode|'+IntToStr(Mode)+'|');
    LogMessage('Global Var-DownloadURL|'+DownloadURL+'|');   
    If (Pos('No results',HTML)<>0) Then Begin                                           //WEB_SPECIFIC
        LogMessage('        No Movies results');
        Result:=prError;
    End Else Begin
        LogMessage('        Parsing search Movies results');
        curPos:=Pos('<div class="lister-list">',HTML);                                 //String which opens the Web Search List data. WEB_SPECIFIC
        //Get first item
        ResultsNumber:=0;
        curPos:=PosFrom('<div class="lister-item mode-simple">',HTML,curPos);            //String which opens the Web Result List data. WEB_SPECIFIC
        While curPos>0 Do Begin
            ResultsNumber:=ResultsNumber+1;
            LogMessage('    Parsing search movie results in '+IntToStr(curPos)+' ...');
            curPos:=PosFrom('<div class="lister-item-image">',HTML,curPos);             //String which opens the Web URL+Title+URL+PreviewURL data. WEB_SPECIFIC
            //Get PreviewURL
            //PreviewURL:='';
            If POSTER_IN_SEARCH Then begin
                ItemValue:=TextBetWeen(HTML,'loadlate="','"',false,curPos);               //Strings which opens/closes the data. WEB_SPECIFIC
                If (Pos('https://m.media-amazon.com/images/G/01/imdb/images/',ItemValue)>0) Then Begin
                    PreviewURL:=TextBetWeenFirst(ItemValue,'https://m.media-amazon.com/images/G/01/imdb/images/','.');     //Get poster code. Strings which opens/closes the data. WEB_SPECIFIC
                    If (Length(PreviewURL)>0) then begin
                        PreviewURL:='https://m.media-amazon.com/images/G/01/imdb/images/' + PreviewURL                        //Base poster URL without '.jpg'. WEB_SPECIFIC
                        ImageFile:=GetAppPath+'Scripts\'+BASE_DOWNLOAD_FILE_IMAGE_NAME+'-SearchResult-'+IntToStr(ResultsNumber)+'-.jpg'
                        // Avoid HTTPS redirection: Download https image to file  OutPutFile=
                        If (1=DownloadImage(PreviewURL + '._V1_UY' + IntToStr(MAX_SEARCHLIST_HEIGTH) + '_.jpg',ImageFile)) then begin  //Dowload in the selected user max size. WEB_SPECIFIC
                            LogMessage('      Parse Results PreviewURL:'+PreviewURL + '._V1_UY' + IntToStr(MAX_SEARCHLIST_HEIGTH) + '_.jpg'+'||');
                            PreviewURL:=ImageFile;  //Pass the downloaded file to PVD
                        end else if (1=DownloadImage(PreviewURL +'.jpg',ImageFile)) then begin  //Dowload in the web base size. WEB_SPECIFIC
                            LogMessage('      Parse Results PreviewURL:' + PreviewURL + '.jpg'+'||');
                            PreviewURL:=ImageFile;  //Pass the downloaded file to PVD
                        end;       
                    end;
                end;
            //end;     
       
            //Get PreviewURL         
            //PreviewURL:='';
            //If POSTER_IN_SEARCH Then begin
                //ItemValue:=TextBetWeen(HTML,'loadlate="','"',false,curPos);               //Strings which opens/closes the data. WEB_SPECIFIC            
                If (Pos(BASE_URL_IMAGE_PRE_TRUE,ItemValue)>0) Then Begin
                    PreviewURL:=TextBetWeenFirst(ItemValue,BASE_URL_IMAGE_PRE_TRUE,'.');     //Get poster code. Strings which opens/closes the data. WEB_SPECIFIC
                    If (Length(PreviewURL)>0) then begin
                        PreviewURL:=BASE_URL_IMAGE_PRE_TRUE + PreviewURL                        //Base poster URL without '.jpg'. WEB_SPECIFIC
                        ImageFile:=GetAppPath+'Scripts\'+BASE_DOWNLOAD_FILE_IMAGE_NAME+'-SearchResult-'+IntToStr(ResultsNumber)+'-.jpg'
                        // Avoid HTTPS redirection: Download https image to file  OutPutFile=
                        If (1=DownloadImage(PreviewURL + '._V1_UY' + IntToStr(MAX_SEARCHLIST_HEIGTH) + '_.jpg',ImageFile)) then begin  //Dowload in the selected user max size. WEB_SPECIFIC
                            LogMessage('      Parse Results PreviewURL:'+PreviewURL + '._V1_UY' + IntToStr(MAX_SEARCHLIST_HEIGTH) + '_.jpg'+'||');
                            PreviewURL:=ImageFile;  //Pass the downloaded file to PVD
                        end else if (1=DownloadImage(PreviewURL +'.jpg',ImageFile)) then begin  //Dowload in the web base size. WEB_SPECIFIC
                            LogMessage('      Parse Results PreviewURL:' + PreviewURL + '.jpg'+'||');
                            PreviewURL:=ImageFile;  //Pass the downloaded file to PVD
                        end;       
                    end;
                end;
            end;
            //Get MovieURL  (Always must it has)
            MovieURL:=TextBetWeen(HTML,'<a href="/title/','/',false,curPos);               //Strings which opens/closes the data. WEB_SPECIFIC
            MovieURL:= BASE_URL_PRE+MovieURL+BASE_URL_SUF;
            LogMessage('      Parse Results URL:'+MovieURL+'||');
            //If true HTPPS link PVdB gives "Socket Error # 11001"
            //If ancient http, get redirected and give "IOHandler value is not valid
            //If invalid url, "HTTP/1.1 400 Bad Request"               
            MovieURL:=CHEAT_PREFIX_URL+MovieURL;                 //'Cheat' URL for avoid HTTPS issue in GET return.
            LogMessage('      Parse Results CHEAT URL:'+MovieURL+'||');
            //Get Title (Always must it has)
            Title:=TextBetWeen(HTML,'>','<',false,curPos);                               //Strings which opens/closes the data. WEB_SPECIFIC
            LogMessage('      Parse Results Title:'+Title+'||');
            //Get Year
            Year:=TextBetWeen(HTML,'(',')',false,curPos);                               //Strings which opens/closes the data. WEB_SPECIFIC
            Year:=StringReplace(Year,'typeof uet == '+Chr(39)+'function'+Chr(39),'',True,False,False);    //Eliminate the CHEAT_PREFIX_URL for get the true movie link           
            LogMessage('    Parse Results in Year:'+Year+'||');
            AddSearchResult(Title,'',Year,MovieURL,PreviewURL);
            //Just to check the website (Only to check the web page) not PVdB valid result.
            if CHECK_WEBSITE then AddSearchResult('                             '+#8729+' Just to check the website: '+StringReplace(MovieURL,CHEAT_PREFIX_URL,'',True,False,False),'','',StringReplace(MovieURL,CHEAT_PREFIX_URL,'',True,False,False),'');
            curPos:=PosFrom('<div class="lister-item mode-simple">',HTML,curPos)          //String which opens the Web Result item List data. WEB_SPECIFIC
        End;
        If ResultsNumber=1 Then Begin
            DownloadURL:=StringReplace(MovieURL,CHEAT_PREFIX_URL,'',True,False,False);    //Eliminate the CHEAT_PREFIX_URL for get the true movie link
            DownloadURL:=StringReplace(DownloadURL,'http:','https:',True,True,False);     //The direct url needed (no https redirection)
            Result:=prDownload; //One result in the DownloadURL.
            LogMessage('      After parsing search Movies,only one result detected');   
        End Else Begin
            Result:=prList; //Don't work with Preferences/Plugings/Silent Mode.
            LogMessage('      After parsing search Movies go to choose List Results');   
        End;
    End;
    LogMessage('Function ParsePage_IMDBSearchTitle END=====================||');
    Exit;
End; //BlockClose


ParsePage_IMDBMovieMPAA

I corrected parts of the code for ParsePage_IMDBMovieMPAA.

Quote
Function ParsePage_IMDBMovieMPAA(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError;  If żany big problem? with exit
    //Retrieve: ~MPAA~
  Var
    curPos:Integer;
    mpaaValue:String;
  Begin
.
.
    LogMessage('      MPAA.USA.Corrected:'+mpaaValue+'||');
   mpaaValue:=StringReplace(mpaaValue,'update: ','',False,False,True);                //WEB_SPECIFIC
   LogMessage('      MPAA rating: '+mpaaValue+' ||');
    //Not or bad MPAA USA then get Spain certification (if exist) and convert in USA-KODI values
.
.
    End;
 
    AddFieldValueXML('mpaa',mpaaValue);
    LogMessage('      Get result MPAA='+mpaaValue+'||');
    LogMessage('Function ParsePage_IMDBMovieMPAA END=====================||');
  End; //BlockClose


Function ParsePage

I added part of the code for Function ParsePage.

Quote
//USER FUNCTIONS AND PROCEDURES==================================================================================BlockClose
//OBLIGATORY CALLBACK PRINCIPAL FUNCTION=========================================================================BlockOpen
Function ParsePage(HTML:String;URL:AnsiString):Cardinal; //BlockOpen
  Var
    MovieID,titleValue,yearValue:String;
    Date:String;
    DateParts: TWideArray;
   
  Begin
    LogMessage('Function ParsePage BEGIN======================|');
    LogMessage('Global Var-Mode|'+IntToStr(Mode)+'|');
    LogMessage('Global Var-DownloadURL|'+DownloadURL+'|');
    LogMessage('Local Var-URL|'+URL+'|');
//Parse return in mode 'Search Movie URL'--------------------------------------------------------------------------------         
.
.
//Parse with the Movie URL 'smNormal'------------------------------------------------------------------------------------
    If (Mode=smNormal) Then Begin
        Result:=prFinished;  //It will change to prError if any big problem with exit; 
        //Information in several page. Needs Provider MovieID
        MovieID:=TextBetWeenFirst(DownloadURL,BASE_URL_PRE_TRUE,BASE_URL_SUF);
        LogMessage('  ParsePage mode smNormal|'+IntToStr(Mode)+'|. Getting provider data for MovieID|'+ MovieID +'|');
        AddCustomFieldValueByName('IMDbID',MovieID);
        LogMessage('      Get result IMDbID:'+MovieID+'||');

        //Get ~url~
        AddFieldValueXML('url',StringReplace(DownloadURL,BASE_URL_PRE_TRUE,BASE_URL_PRE,True,False,False));
        LogMessage('      Get result url:'+StringReplace(DownloadURL,BASE_URL_PRE_TRUE,BASE_URL_PRE,True,False,False)+'||');
    //Parse principal provider page = BASE_URL_PRE_TRUE------------------------------------------------------------------
.
.         
    //Date ~Updated~ (choose simple or verbose version)
        Date:=DateToStr(CurrentDateTime);
        if Date <> '' then
         begin
            ExplodeString(Date,DateParts,'-');
            Date:=DateParts[2]+'.'+ DateParts[1]+'.'+DateParts[0];   
            AddCustomFieldValueByName('Updated',Date+' at '+TimeToStr(CurrentDateTime)+' • '+SCRIPT_NAME+' '+SCRIPT_VERSION);  // Annoying 
        End;   
      
        LogMessage('    Provider data info retreived Ok in '+DateToStr(CurrentDateTime)+' '+TimeToStr(CurrentDateTime)+'|');
        LogMessage('Function ParsePage NORMAL END======================|');
        exit;
    End;
//Parse with unknow mode-----------------------------------------------------------------------------------------------   
    Result:=prError;
    exit;
End; //BlockClose
//OBLIGATORY CALLBACK PRINCIPAL FUNCTION=========================================================================BlockClose   
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 02, 2018, 09:21:50 am
IMDB_ [EN] [HTTPS] script

I added and Corrected part of the code for Function ParsePage_IMDBMovieBASE to now work properly.

Part 1:

//SCRIPT INFO

Quote
//SCRIPT INFO================================================================================================== BlockClose
//SCRIPT CONSTANTS============================================================================================= BlockOpen
Const
//Script types-----------------------------------------------------------------------------------------------------------
  stMovies        = 0;
  stPeople        = 1;
  stPoster        = 2;
  stScreenShots   = 3;
  stCover         = 4;
//Script Obligatory Infos----------------------------------------------------------------------------------------------
  SCRIPT_NAME  = 'IMDB [EN][HTTPS]_1'; //The most complete information provider
  SCRIPT_DESC  = 'IMDB download Movie info [EN]_1'; 
  SCRIPT_VERSION  = '1.1.0.0';
  SCRIPT_TYPE  = stMovies;
  SCRIPT_LANG  = $09; //English //$0a es-Spanish   $0c fr-French   $09 en-English
  RATING_NAME    = 'IMDB'; //PVD has ~imdbrating~ for this value
  CODE_PAGE      = 65001; //28591=ISO 8859-1 Latin 1; Western European (ISO). Use: 65001=Unicode (UTF-8) | 0=for Autodetect   Autodetect   
//Script Options-------------------------------------------------------------------------------------------------------
  //Retreive Data Config
  GET_MAINPAGE_ACTORS_CREDIT  = True ;  //Set to True to ensure MAINPAGE_ACTORS_CREDIT info is added. Otherwise: Set to False to ensure MAINPAGE_ACTORS_CREDIT info is not added. MAINPAGE_ACTORS_CREDIT info is in to the principal movie page.
  GET_FULL_ACTORS_CREDIT  = True ;  //Set to True to ensure FULL_ACTORS_CREDIT info is for Full Cast & Crew provider page. Otherwise: Set to False to ensure FULL_ACTORS_CREDIT info is not added.
  GET_FULL_MPAA  = True ;  //Download Certification provider page for retreive the MPAA info. Otherwise only the info of the principal movie page.
  GET_FULL_AKA  = True ;  //Download 'Also Known As' provider page for retreive the info. Otherwise only the info of the principal movie page.
  GET_FULL_CREDIT  = True ;  //Download Cast or Credit provider page for retreive the info. Otherwise only the info of the principal movie page.
  PEOPLE_LIMIT  = 25;    //Limit of number of actors (cast) or of credits (crew) retrieved.
  GET_FULL_AWARDS  = True ;  //Download Awards provider page for retreive the info. Otherwise doesn't do nothingh because no info in the principal movie page.
  EVENTS_LIMIT  = 100;   //Limit of number of events (USA Academy Awards, Golden Globes, etc) to retrive awards.
  GET_FEATURES  = True ; //Retreive the original movie features (Runtime, Sound Mix, Color, Aspect Ratio).
  GET_FULL_FEATURES  = True ; //Download Technical Specs for retreive the original movie features (Runtime, Sound Mix, Color, Aspect Ratio, etc).
  MAX_IMAGE_HEIGTH  = 1200; //Heigth limit of the stored posters.
  //Process Data Config
.
.
//Download methods-------------------------------------------------------------------------------------------------------
  dmGET           = 0;
  dmPOST          = 1;
//Movie fields----------------------------------------------------------------------------------------------------------
   mfURL      = 0;
   mfTitle      = 1;
   mfOrigTitle   = 2;
   mfAka      = 3;
   mfYear      = 4;
   mfGenre      = 5;
   mfCategory   = 6;
   mfCountry   = 7;
   mfStudio      = 8;
   mfMPAA      = 9;
   mfRating      = 10;   //This is 'Additional rating', not 'Rating'
   mfTags      = 11;
   mfTagline   = 12;
   mfDescription   = 13;
   mfDuration   = 14;
   mfFeatures   = 15;

//Credits types----------------------------------------------------------------------------------------------------------

Function ParsePage_IMDBMovieBASE

Quote
Function ParsePage_IMDBMovieBASE(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit;
    //Retrieve: ~title~, ~year~, ~origtitle~, ~poster~ / ~imdbrating~, ~IMDB_Votes~ (Custom Field) / ~TOP_250~(Custom Field) /
    //          If Not(GET_FULL_CREDIT): ~crew~ctDirectors,ctWriters,ctComposers,ctProducers(Not in base page), ctActors
    //         ~description~ / ~category~ "keywords" / ~tagline~ / ~genre~
    //         If Not(GET_FULL_MPAA) ~mpaa~
    //         ~country~ / ~rdate~ in contry provider local IP geolocation
    //         If Not(GET_FULL_AKA) ~aka~.
    //         ~budget~ / ~money~ / ~studio~ "Production Co"
   //         If GET_FEATURES ~features~   
    //         //If Not(GET_FULL_FEATURES) ~features~
  Var
      curPos,endPos,index:Integer;
      ItemValue,ItemList,ImageFile:String;
      titleValue:String;
      Name,Role,PersonURL:String;
  Begin
    LogMessage('Function ParsePage_IMDBMovieBASE BEGIN======================|');
    Result:=prFinished;  //It will change to prError if any big problem with exit;
    //Because the script don't retreives the data in the page order a token search for the first curPos position or block select is mandatory
    //Get ~title~, ~year~, ~origtitle~, ~poster~
    //Get all "raw" title summary (in raw because we need the hidden links, we avoid "complete" token in strings which opens/closes)
    //ItemList:=TextBetWeenFirst(HTML,'<div class="title_wrapper"','div class="plot_summary_wrapper">'); //WEB_SPECIFIC.
    ItemList:=TextBetWeenFirst(HTML,'<div class="title_block"','div class="plot_summary_wrapper">'); //WEB_SPECIFIC.   
    If (Length(ItemList)>0) Then Begin           
        //titleValue:=TextBetWeenFirst(ItemList,'<h1 itemprop="name" class="">','<');       //Strings which opens/closes the data. WEB_SPECIFIC
        titleValue:=TextBetWeenFirst(ItemList,'<h1 class="">','<');       //Strings which opens/closes the data. WEB_SPECIFIC      
        AddFieldValueXML('title',titleValue);
        LogMessage('      Get result title:'+titleValue+'||');
        ItemValue:=TextBetWeenFirst(ItemList,'(','–');                                 //Strings which opens/closes the data. WEB_SPECIFIC   
        LogMessage('      Get result year_00:'+TextBetWeenFirst(ItemList,'(','–')+'||');           
        AddFieldValueXML('year',ItemValue);
        LogMessage('      Get result year_0:'+ItemValue+'||');   
   
        //ItemValue:=TextBetWeenFirst(ItemList,'(',')');                                 //Strings which opens/closes the data. WEB_SPECIFIC
        ItemValue:=TextBetWeenFirst(ItemList,'<span id="titleYear">(',')</span>            </h1>');                                 //Strings which opens/closes the data. WEB_SPECIFIC      
        AddFieldValueXML('year',ItemValue);
        LogMessage('      Get result year:'+ItemValue+'||');
        ItemValue:=TextBetWeenFirst(ItemList,'<div class="originalTitle">','<');          //Strings which opens/closes the data. WEB_SPECIFIC
        If (Length(ItemValue)=0) Then ItemValue:=titleValue;                           //Provider hides the original title if same that title. WEB_SPECIFIC
        AddFieldValueXML('origtitle',ItemValue);
        LogMessage('      Get result origtitle:'+ItemValue+'||');
        ItemValue:=TextBetWeenFirst(ItemList,BASE_URL_IMAGE_PRE_TRUE,'.');                 //Get poster code. Strings which opens/closes the data. WEB_SPECIFIC
        If (Length(ItemValue)>0) then begin
            ItemValue:=BASE_URL_IMAGE_PRE_TRUE + ItemValue;                             //Base poster URL without '.jpg'. WEB_SPECIFIC
            ImageFile:=GetAppPath+'Scripts\'+BASE_DOWNLOAD_FILE_IMAGE_NAME+'-Poster.jpg'
            // Avoid HTTPS redirection: Download https image to file  OutPutFile=
            If (1=DownloadImage(ItemValue + '._V1_UY' + IntToStr(MAX_IMAGE_HEIGTH) + '_.jpg',ImageFile)) then begin  //Dowload in the selected user max size. WEB_SPECIFIC
                AddImageURL(itPoster,ImageFile);    //Get the photo from a file even if the next line we log the true URL.
                LogMessage('      Get result poster:'+ItemValue + '._V1_UY' + IntToStr(MAX_IMAGE_HEIGTH) + '_.jpg'+'||');
            end else if (1=DownloadImage(ItemValue +'.jpg',ImageFile)) then begin  //Dowload in the web base size. WEB_SPECIFIC
                AddImageURL(itPoster,ImageFile);    //Get the photo from a file even if the next line we log the true URL.
                LogMessage('      Get result poster:' + ItemValue + '.jpg'+'||');
            end;       
        end;
    End;
    //Get ~imdbrating~, ~IMDB_Votes~
    //curPos:=Pos('<span itemprop="ratingValue">',HTML);                      //WEB_SPECIFIC.
    curPos:=Pos('<div class="ratingValue">',HTML);                      //WEB_SPECIFIC.
    If 0<curPos Then Begin                         
       //ItemValue:=TextBetWeen(HTML,'<span itemprop="ratingValue">','<',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=TextBetWeen(HTML,'<span>','<',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=StringReplace(ItemValue,',','.',True,True,False);                           //Decimal comma spanish separator to point english separator.
       AddFieldValueXML('imdbrating',ItemValue);
       LogMessage('      Get result imdbrating:'+ItemValue+'||');
       //ItemValue:=TextBetWeen(HTML,'<span class="small" itemprop="ratingCount">','</span>',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=TextBetWeen(HTML,'<span class="small">','</span>',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC      
       ItemValue:=StringReplace(ItemValue,'.','',True,True,False);            //Remove thousand point spanish separator.
       ItemValue:=StringReplace(ItemValue,',','',True,True,False);            //Remove thousand comma english separator.
       AddCustomFieldValueByName('IMDB_Votes',ItemValue);
       LogMessage('      Get result IMDB_Votes:'+ItemValue+'||');
    End;
    //Get ~TOP_250~
    curPos:=Pos('Top Rated Movies #',HTML);                                 //WEB_SPECIFIC.
    If 0<curPos Then Begin
       ItemValue:=TextBetWeen(HTML,'Top Rated Movies #','<',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
       AddCustomFieldValueByName('TOP_250',ItemValue);
       LogMessage('      Get result TOP_250:'+ItemValue+'||');
    End;
    //Get ~Bottom 100~
    curPos:=Pos('Bottom Rated Movies #',HTML);                                 //WEB_SPECIFIC.
    If 0<curPos Then Begin
       ItemValue:=TextBetWeen(HTML,'Bottom Rated Movies #','<',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
       AddCustomFieldValueByName('Bottom 100',ItemValue);
       LogMessage('      Get result Bottom_100:'+ItemValue+'||');
    End;
    //~Metascore:~
    curPos := PosFrom('> <div class="metacriticScore score_favorable titleReviewBarSubItem">',HTML,EndPos);
    if curPos>0 then begin
        curPos:=curPos+Length('> <div class="metacriticScore score_favorable titleReviewBarSubItem">');
        EndPos:=PosFrom('</span>',HTML,curPos);
        ItemValue:=RemoveTags(Copy(HTML,curPos,endPos-curPos+2),false);
        ItemValue:=StringReplace(ItemValue,'0</',',0',True,False,True);
        ItemValue:=StringReplace(ItemValue,'1</',',1',True,False,True);
        ItemValue:=StringReplace(ItemValue,'2</',',2',True,False,True);
        ItemValue:=StringReplace(ItemValue,'3</',',3',True,False,True);
        ItemValue:=StringReplace(ItemValue,'4</',',4',True,False,True);
        ItemValue:=StringReplace(ItemValue,'5</',',5',True,False,True);
        ItemValue:=StringReplace(ItemValue,'6</',',6',True,False,True);
        ItemValue:=StringReplace(ItemValue,'7</',',7',True,False,True);
        ItemValue:=StringReplace(ItemValue,'8</',',8',True,False,True);
        ItemValue:=StringReplace(ItemValue,'9</',',9',True,False,True);
            //AddFieldValueXML('orname',RATING_NAME1);
            //AddFieldValueXML('orating',ItemValue);   
        AddCustomFieldValueByName('Metascore',FloatToStr((StrToFloat(ItemValue) * 1)));
    End;
    //~Metascore:~
    curPos:=PosFrom('> <div class="metacriticScore score_unfavorable titleReviewBarSubItem">',HTML,EndPos);
    if curPos>0 then begin
        curPos:=curPos+Length('> <div class="metacriticScore score_unfavorable titleReviewBarSubItem">');
        EndPos:=PosFrom('</span>',HTML,curPos);
        ItemValue:=RemoveTags(Copy(HTML, curPos,endPos-curPos+2),false);
        ItemValue:=StringReplace(ItemValue,'0</',',0',True,False,True);
        ItemValue:=StringReplace(ItemValue,'1</',',1',True,False,True);
        ItemValue:=StringReplace(ItemValue,'2</',',2',True,False,True);
        ItemValue:=StringReplace(ItemValue,'3</',',3',True,False,True);
        ItemValue:=StringReplace(ItemValue,'4</',',4',True,False,True);
        ItemValue:=StringReplace(ItemValue,'5</',',5',True,False,True);
        ItemValue:=StringReplace(ItemValue,'6</',',6',True,False,True);
        ItemValue:=StringReplace(ItemValue,'7</',',7',True,False,True);
        ItemValue:=StringReplace(ItemValue,'8</',',8',True,False,True);
        ItemValue:=StringReplace(ItemValue,'9</',',9',True,False,True);
            //AddFieldValueXML('orname',RATING_NAME1);
            //AddFieldValueXML('orating',ItemValue);   
        AddCustomFieldValueByName('Metascore',FloatToStr((StrToFloat(ItemValue) * 1)));
    End;
   
    //Get Cast. GET_FULL_CREDIT = False only the info of the principal movie page.
.
.
    //Get ~plot_summary~
    If Pos('<div class="plot_summary',HTML)>0 then begin
    EndPos:=Pos('<div class="plot_summary',HTML);
    curPos:=Pos('<div class="summary_text">',HTML);                                 //WEB_SPECIFIC.
       If 0<curPos Then Begin
          ItemValue:=TextBetWeen(HTML,'<div class="summary_text">','</div>',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
          ItemValue:=StringReplace(ItemValue,',            ',', ',True,False,True);
          ItemValue:=StringReplace(ItemValue,'            ',' ',True,False,True);
          ItemValue:=StringReplace(ItemValue,'        ',' ',True,False,True);
          ItemValue:=StringReplace(ItemValue,'Add a Plot »','',True,False,True);
          ItemValue:=StringReplace(ItemValue,'See full summary »','',True,False,True);
          curPos:=Pos('var ue_t0=ue_t0||+new Date();var ',ItemValue);
          If 0<curPos then ItemValue:=Copy(ItemValue,0,curPos-1);
          AddFieldValueXML('comment',ItemValue);
          LogMessage('      Get result plot_summary description:'+ItemValue+'||');
       End;
    End;[/b]
.
.
 
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 02, 2018, 09:27:30 am
IMDB_ [EN] [HTTPS] script

Part 2:

Function ParsePage_IMDBMovieBASE

Quote

    //Get ~plot_summary~
    If Pos('<div class="plot_summary',HTML)>0 then begin
    EndPos:=Pos('<div class="plot_summary',HTML);
    curPos:=Pos('<div class="summary_text">',HTML);                                 //WEB_SPECIFIC.
       If 0<curPos Then Begin
          ItemValue:=TextBetWeen(HTML,'<div class="summary_text">','</div>',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
          ItemValue:=StringReplace(ItemValue,',            ',', ',True,False,True);
          ItemValue:=StringReplace(ItemValue,'            ',' ',True,False,True);
          ItemValue:=StringReplace(ItemValue,'        ',' ',True,False,True);
          ItemValue:=StringReplace(ItemValue,'Add a Plot »','',True,False,True);
          ItemValue:=StringReplace(ItemValue,'See full summary »','',True,False,True);
          curPos:=Pos('var ue_t0=ue_t0||+new Date();var ',ItemValue);
          If 0<curPos then ItemValue:=Copy(ItemValue,0,curPos-1);
          AddFieldValueXML('comment',ItemValue);
          LogMessage('      Get result plot_summary description:'+ItemValue+'||');
       End;
    End;

.
.
    //Get ~category~ "keywords" (field with several values in a comma separated list)
    curPos:=Pos('<h4 class="inline">Plot Keywords:</h4>',HTML);                                      //WEB_SPECIFIC.IC.
    If 0<curPos Then Begin       
       ItemValue:=TextBetWeen(HTML,'<h4 class="inline">Plot Keywords:</h4>','<a href="/title/',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=StringReplace(ItemValue,'|',',',True,True,False);                                  //Change the separator to comma
       //AddFieldValueXML('category',ItemValue);
      AddFieldValueXML('tags',ItemValue);       
       LogMessage('      Get results Category:'+ItemValue+'||');
    End;
.
.
    //Get ~mpaa~. GET_FULL_MPAA = False only the info of the principal movie page.
    If Not(GET_FULL_MPAA) Then Begin
       //The text can be "Certificate:" or "Motion Picture Rating" but alway after genres.
       If 0<curPos Then Begin
          ItemValue:=TextBetWeen(HTML,'<span itemprop="contentRating">','</span>',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
          AddFieldValueXML('mpaa',ItemValue);
          LogMessage('      Get result mpaa:'+ItemValue+'||');
       End;
    End;
    //Get ~origlang~ (several values in a comma separated list)
    curPos:=Pos('<h4 class="inline">Language:</h4>',HTML);                                      //WEB_SPECIFIC.
    if 0<curPos then begin       
       ItemValue:=TextBetWeen(HTML,'<h4 class="inline">Language:</h4>','</div>',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
       curPos:=Pos('            |',ItemValue);
       If 0<curPos then ItemValue:=Copy(ItemValue,0,curPos-1);     
       AddFieldValueXML('origlang',ItemValue);
       LogMessage('      Get results Language:'+ItemValue+'||');
    End;

    //Get ~country~ (several values in a comma separated list)
    curPos:=Pos('<h4 class="inline">Country:</h4>',HTML);                                      //WEB_SPECIFIC.
    If 0<curPos Then Begin       
       ItemValue:=TextBetWeen(HTML,'<h4 class="inline">Country:</h4>','</div>',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=StringReplace(ItemValue,'|',',',True,True,False);                           //Change the separator to comma
       AddFieldValueXML('country',ItemValue);
       LogMessage('      Get results Country:'+ItemValue+'||');
    End;
    //Get ~rdate~ in contry provider local IP geolocation. See: http://sobizarre-en.blogspot.fr/2014/12/how-to-easily-defeat-imdb-geolocation.html
    curPos:=Pos('<h4 class="inline">Release Date:</h4>',HTML);                                      //WEB_SPECIFIC
    If 0<curPos Then Begin       
        ItemValue:=TextBetWeen(HTML,'<h4 class="inline">Release Date:</h4>','(',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
        ItemValue:=StringReplace(ItemValue,'January','1',true,true,true);     //Format web corrections. WEB_SPECIFIC
        ItemValue:=StringReplace(ItemValue,'February','2',true,true,true);
        ItemValue:=StringReplace(ItemValue,'March','3',true,true,true);
        ItemValue:=StringReplace(ItemValue,'April','4',true,true,true);
        ItemValue:=StringReplace(ItemValue,'May','5',true,true,true);
        ItemValue:=StringReplace(ItemValue,'June','6',true,true,true);
        ItemValue:=StringReplace(ItemValue,'July','7',true,true,true);
        ItemValue:=StringReplace(ItemValue,'August','8',true,true,true);
        ItemValue:=StringReplace(ItemValue,'September','9',true,true,true);
        ItemValue:=StringReplace(ItemValue,' October ',' 10 ',true,true,true);
        ItemValue:=StringReplace(ItemValue,' November ',' 11 ',true,true,true);
        ItemValue:=StringReplace(ItemValue,' December ',' 12 ',true,true,true);
            //ItemValue:=StringReplace(ItemValue,' ','.',True,False,True);
        ItemValue:=StringReplace(ItemValue,' ',#46,True,False,True);
        ItemValue:=StringReplace(ItemValue,'October.','10.',True,True,True);
        ItemValue:=StringReplace(ItemValue,'November.','11.',True,True,True);       
        ItemValue:=StringReplace(ItemValue,'December.','12.',True,True,True);     //Format web corrections. WEB_SPECIFIC

        AddFieldValueXML('rdate',ItemValue);
        LogMessage('      Get results Release Date:'+ItemValue+'||');
    End;       
    //Get ~aka~. GET_FULL_AKA = False only the info of the principal movie page.       
    If Not(GET_FULL_AKA) Then Begin
       curPos:=Pos('<h4 class="inline">Also Known As:</h4>',HTML);                                      //WEB_SPECIFIC
       If 0<curPos Then Begin
          ItemValue:=TextBetWeen(HTML,'<h4 class="inline">Also Known As:</h4>','<',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
          AddFieldValueXML('aka',ItemValue);
          LogMessage('      Get result aka:'+ItemValue+'||');
       End;
    End;
    //Get ~budget~
    curPos:=Pos('<h4 class="inline">Budget:</h4>',HTML);                                      //WEB_SPECIFIC
    If 0<curPos Then Begin       
       ItemValue:=TextBetWeen(HTML,'<h4 class="inline">Budget:</h4>','<',false,curPos);      //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=StringReplace(ItemValue,#36,'',True,True,False);        //Eliminate '$' if exists 
       ItemValue:=StringReplace(ItemValue,'€','',True,True,False);        //Eliminate '€' if exists         
       ItemValue:=StringReplace(ItemValue,',','',True,True,False);        //Eliminate ',' if exists             
       AddFieldValueXML('budget',ItemValue);
       LogMessage('      Get results Budget:'+ItemValue+'||');
    End;       
    //Get ~money~ Box Office worldwide     
.
.
    //Get ~studio~ "Production Co" (several values in a comma separated list)
    curPos:=Pos('<h4 class="inline">Production Co:</h4>',HTML);                                      //WEB_SPECIFIC.IC.
    If 0<curPos Then Begin       
       ItemValue:=TextBetWeen(HTML,'<h4 class="inline">Production Co:</h4>','</span>',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=StringReplace(ItemValue,'See more »','',True,False,True);      
       ItemValue:=StringReplace(ItemValue,'See more','',True,True,False);                           //Eliminate 'See more' if exists           
       ItemValue:=StringReplace(ItemValue,', The','',True,False,True);
       ItemValue:=StringReplace(ItemValue,'The, ','',True,False,True);
       //ItemValue:=StringReplace(ItemValue,'The ','',True,False,True);   
   
       AddFieldValueXML('studio',ItemValue);
       LogMessage('      Get results Studio/Production Co:'+ItemValue+'||');
    End;
    //Get ~features~ (multiline) GET_FULL_FEATURES = False only the info of the principal movie page.
    //If Not(GET_FULL_FEATURES) Then Begin
    If GET_FEATURES Then Begin
        ItemList:='';
        //Get "Runtime" (in minutes)
        curPos:=Pos('<h4 class="inline">Runtime:</h4>',HTML);                                      //WEB_SPECIFIC.
        If 0<curPos Then Begin 
            ItemValue:=TextBetWeen(HTML,'M">',' min</time>',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
            ItemList:=ItemList+'Original Runtime: '+ItemValue+' min.<br>';
            LogMessage('      Get result Original Runtime:'+ItemValue+'||');
            AddFieldValue(mfDuration,ItemValue);
            LogMessage('      Get result lenght:'+ItemValue+'||');
        End;   
.
.
        if (Length(ItemList)>0) then begin
            AddFieldValueXML('features',ItemList);
            LogMessage('      Get result Movie Features:'+ItemList+'||');
        End;
    End;
    //Get ~User Reviews~
    If Pos('<h2>User Reviews</h2>',HTML)>0 then begin
    EndPos:=Pos('<h2>User Reviews</h2>',HTML);
    curPos:=Pos('<h2>User Reviews</h2>',HTML);                                 //WEB_SPECIFIC.
       If 0<curPos Then Begin
        curPos:=PosFrom('<p>',HTML,EndPos)+Length('<p>');
          EndPos:=PosFrom('</span>',HTML,curPos);
          ItemValue:=Copy(HTML,curPos,endPos-curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
        ItemValue:=StringReplace(ItemValue,'</p>','',True,False,True);
        ItemValue:=StringReplace(ItemValue,'</div>','',True,False,True);
        ItemValue:=StringReplace(ItemValue,'<ul><li>','',True,False,True);            
        ItemValue:=StringReplace(ItemValue,'</li></ul>','',True,False,True);
        ItemValue:=StringReplace(ItemValue,':)','',True,False,True);   
        curPos:=Pos('Find showtimes, ',ItemValue);
        If 0<curPos then ItemValue:=Copy(ItemValue,0,curPos-1);
          AddCustomFieldValueByName('User Comments',ItemValue);
        AddCustomFieldValueByName('User Reviews',ItemValue);
          LogMessage('      Get result User Reviews: <br>'+ItemValue+'||');
       End;
    End;
      
    LogMessage('Function ParsePage_IMDBMovieBASE END=====================||');
  End; //BlockClose

//Get ~category~ "keywords"

Quote
    //Get ~category~ "keywords" (field with several values in a comma separated list)
    curPos:=Pos('<h4 class="inline">Plot Keywords:</h4>',HTML);                                      //WEB_SPECIFIC.IC.
    If 0<curPos Then Begin       
       ItemValue:=TextBetWeen(HTML,'<h4 class="inline">Plot Keywords:</h4>','<a href="/title/',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=StringReplace(ItemValue,'|',',',True,True,False);                                  //Change the separator to comma
       //AddFieldValueXML('category',ItemValue);
      AddFieldValueXML('tags',ItemValue);       
       LogMessage('      Get results Category:'+ItemValue+'||');
    End;

As for ~ category ~ "keywords" IMDB has never, explicitly repeat I never knew the ~ category ~ "keywords" as they say, have the AllMovie site. This has always been taken as ~ tags ~ "keywords" and information has always been transferred to the Tags field in dababase.

For example for 21 movie:

IMDB "keywords"

Quote
Plot Keywords: based on true story | card counting | blackjack | massachusetts institute of technology | gambling | See All (109) »
https://www.imdb.com/title/tt0478087/

AllMovie "keywords"
https://www.allmovie.com/movie/21-v386347

Quote
Keywords
betrayal, blackjack [card game], casino, college-student
AllMovie "category"
Quote
Themes
Gambling | Schemes and Ruses | Teachers and Students | Betrayal | Crime Gone Awry | Dishonor Among Thieves | Perfect Crime
https://www.allmovie.com/movie/21-v386347

It has been used since the creation of the PVD program, or it has been almost ten years since, for this I know that it was added to the database
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 02, 2018, 09:53:06 am
IMDB_ [EN] [HTTPS] script

//Get ~features~ (multiline)

Quote
    //Get ~features~ (multiline) GET_FULL_FEATURES = False only the info of the principal movie page.
    //If Not(GET_FULL_FEATURES) Then Begin
    If GET_FEATURES Then Begin
        ItemList:='';
        //Get "Runtime" (in minutes)
        curPos:=Pos('<h4 class="inline">Runtime:</h4>',HTML);                                      //WEB_SPECIFIC.
        If 0<curPos Then Begin 
            ItemValue:=TextBetWeen(HTML,'M">',' min</time>',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
            ItemList:=ItemList+'Original Runtime: '+ItemValue+' min.<br>';
            LogMessage('      Get result Original Runtime:'+ItemValue+'||');
            AddFieldValue(mfDuration,ItemValue);
            LogMessage('      Get result lenght:'+ItemValue+'||');
        End;   
.
.
        if (Length(ItemList)>0) then begin
            AddFieldValueXML('features',ItemList);
            LogMessage('      Get result Movie Features:'+ItemList+'||');
        End;
    End;

Here is a change because this is a bit different information than in the Technical Specifications section.

Get ~features~

I've repaired parts of the Information Transfer Code for the Technical Specifications section.

Quote
Function ParsePage_IMDBMovieTECHNICAL(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit
    //Retrieve: ~features~
  Var
    //curPos:Integer;
    curPos,endPos:Integer;
    ItemValue,ItemList:String;     
  Begin
.
.
    //Get "Runtime" info
    ItemValue:=HTMLValues2(HTML,'Runtime','</tr>','<td>','</tr>','<br><br>',EndPos);
    ItemValue:=StringReplace(ItemValue,'                                ','<br>',True,False,True);
    ItemValue:=StringReplace(ItemValue,'                      ','<br>',True,False,True);
    ItemValue:=StringReplace(ItemValue,'          ','',True,False,True);
    LogMessage('      Get result Runtime:'+ItemValue+'||');
    if ItemValue <> '' then ItemList:=ItemList+'Runtime'+'<br>'+ItemValue+'<br><br>'; 
         curPos:=Pos('<td class="label"> Runtime </td>',HTML);                                      //WEB_SPECIFIC.
         If 0<curPos Then Begin 
            ItemValue:=TextBetWeen(HTML,' (',' min)',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
            AddFieldValue(mfDuration,ItemValue);
            //AddFieldValueXML('lenght',ItemValue);
            LogMessage('      Get result Original Runtime_1:'+ItemValue+'||');
         End;      
   
.
.
    //Get "Color" info
    ItemValue:=HTMLValues2(HTML,'Color','</tr>','<td>','</tr>','<br><br>',EndPos);
    ItemValue:=StringReplace(ItemValue,'                     |          ',' • ',True,False,True);
    ItemValue:=StringReplace(ItemValue,'             |         ',' • ',True,False,True);
    ItemValue:=StringReplace(ItemValue,'        ',' ',True,False,True);
   ItemValue:=StringReplace(ItemValue,'  • ',' • ',True,False,True);
    LogMessage('      Get result Color:'+ItemValue+'||');
    if ItemValue <> '' then ItemList:=ItemList+'Color'+'<br>'+ItemValue+'<br><br>';
.
.         
    if (Length(ItemList)>0) then begin
        //AddFieldValueXML('features',ItemList);   //Ivek23 CustomField ~ImdbTechSpecs~
      AddCustomFieldValueByName('ImdbTechSpecs',ItemList);
        LogMessage('      Get result Movie Features:'+ItemList+'||');
    End;
    LogMessage('Function ParsePage_IMDBMovieTECHNICAL END=====================||');
  End; //BlockClose

The Technical Specifications section information is recommended to be transferred to the ~ ImdbTechSpecs ~ custom field.

IMDB_ [EN] [HTTPS] _1 script is attached.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: fmb64 on September 08, 2018, 01:28:53 pm
Is it normal that importing from IMDb is very slow and screen is flickering from quickly disappearing black windows... also importing does hang when I run it from C:\Program Files\... instead of a temporary directory?
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 08, 2018, 03:36:47 pm
Is it normal that importing from IMDb is very slow and screen is flickering from quickly disappearing black windows... also importing does hang when I run it from C:\Program Files\... instead of a temporary directory?

Yes, because the external program must first download all the information from a webpage to a temp file so that it can then be added to the database for a particular movie. Because more than information for a particular movie on web pages and is written there, it takes longer for the external program to passes it, as previously mentioned.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: fmb64 on September 09, 2018, 08:16:01 am
Ok.

Is it possible to hide the flickering windows or run them minimized? Very annoying when you want wo work with another window/software while importing. Even when I switch to a full screen game, it jumps back to the desktop ;)

Also the PVD window gets bigger and overlaying the taskbar while importing.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on September 09, 2018, 01:52:45 pm
Quote
Is it possible to hide the flickering windows or run them minimized?
unfortunately not because the PVD makes the launch. The MOD tries to hide it and recover the focus of windows PVD, but it can not be done more.
Quote
Is it normal that importing from IMDb is very slow ...
As Ivek23 responded, the IMDB script can arrive to download to 6 pages per movie. In addition, IMDB increase the response time when it detects multiple queries of an IP user.
Have you tried the TheMovieDB script? This free provider is specially designed for this type of programs: download only the precise information in text mode, so it works very fast.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on September 09, 2018, 02:25:35 pm
Ivek23 you had done an extraordinary work. It will take me some weeks to put in in a new MOD version.

I have one question in order to set un standart for ~genre~ / ~category~ / ~tags~. I try to summary:
~genre~ = Present as "genre" in IMDB, TheMovieDB, FilmAffinity,AllMovie
~tags~   = Present as "keywords" in IMDB, AllMovie, TheMovieDB
~category~ = Not present in IMDB neither TheMovieDB but
     FilmAffinity have the information "Genre1.Genre2 | Category1.Category2"
     AllMovie have the information Theme1 | Theme2 | Theme3 | Theme4 |
Are you agree?

I have other questions over the custom fields but I must study one by one.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 11, 2018, 06:28:48 am
Ivek23 you had done an extraordinary work. It will take me some weeks to put in in a new MOD version.

Thanks.

I have one question in order to set un standart for ~genre~ / ~category~ / ~tags~. I try to summary:
~genre~ = Present as "genre" in IMDB, TheMovieDB, FilmAffinity,AllMovie
~tags~   = Present as "keywords" in IMDB, AllMovie, TheMovieDB

Yes, correct reasoning.

~category~ = Not present in IMDB neither TheMovieDB but

The same for this.

     FilmAffinity have the information "Genre1.Genre2 | Category1.Category2"

For this, I can not say anything because I do not use it.

     AllMovie have the information Theme1 | Theme2 | Theme3 | Theme4 |

Yes, but there are several of these categories under different names. It will be necessary to list the names of these, so that I can answer this in more detail.

I have other questions over the custom fields but I must study one by one.

With questions on this problem, when you need it.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on September 13, 2018, 09:55:30 pm
Attached the files and scripts with the next changelog:
IMDB_[EN][HTTPS]
        V 1.2.0.0 (11/09/2018) VVV: PVdBScriptConfig.ini -> Bug corrected "GET_FULL_FEATURES"
           Ivek23: ParsePage_IMDBMovieCREDIT (Full Cast and Crew section) = Repaired pieces of code for transmitting actors information
           Ivek23: ParsePage_IMDBSearchTitle = Corrected missing posters in search movies results.
           Ivek23: ParsePage_IMDBMovieMPAA = Corrected parts of the code.
           Ivek23: ParsePage = Added compatibility with CustomFields ~IMDbID~, ~Updated~ (last needed because is not possible save in ~moddate~).
           Ivek23,VVV: ParsePage_IMDBMovieBASE = Corrected to the API evolution. Now get "keywords" in ~tags~ (not in ~category~). Get ~origlang~.
                       Added compatibility with CustomFields ~Bottom 100~, ~Metascore~, ~User Reviews~
                      New script parameter PRESERVE-PVD-DESCRIPTION-SETTING = Avoid double use of ~description~ and ~comment~ using only the user check fields table selection otherwise try get "Storyline" and "plot_Summary".
           Ivek23,VVV: New script parameter PRESERVE-USER-MOVIE-CHARS = Preserve the user movie copy characteristics (length and languages channels) instead of the original runtime and languages.
           Ivek23,VVV: ParsePage_IMDBMovieTECHNICAL = Added compatibility with CustomFields ~ImdbTechSpecs~ (original PVD ~features~ can be disabled in the program check fields table)
TheMovieDB_[XX][API]
        V 1.0.0.1 (11/09/2018)  VVV: TheMovieDB "keywords" info goes to ~tags~ PVD field (Ivek23 suggestion). Added ~Updated~ field.
FilmAffinity_[XX][HTTPS]
            V 4.0.0.4 (11/09/2018) VVV: Added ~Updated~ field

Some commentaries:
It will be added the custom field ~Updated~(Short text)  to the standard database because the PVD give problems with the ~moddate~ but the other custom fields, even if the scripts write the information, will not be added because are provider oriented.
There are two database uses:
1) the use at the same time of ~description~ and ~comments~
2) set the original lengh movie and lenguage that over the user file movie copy
that are not standart. Now there is two script parameters to allow the user this behavior.

Ivek23, may you test the script in order to verify if it work as you expect even with your custom fields?
When all work I' ll publish a new MOD version. Please, as me if you have any doubt.  ;D
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 14, 2018, 07:20:05 am
Ivek23, may you test the script in order to verify if it work as you expect even with your custom fields?

I will test, it will take a few days or a week or two, all depends on my free time, which unfortunately I do not have much at the moment. It can also be anything sooner, we will see, because I have found some more errors that I have repaired and checked if they work properly.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on September 16, 2018, 08:28:04 am
Today the scripts configuration parameters are hardcode in the script. This is not a good practice (and BTW is the reason why PVD must restart after use **** Scripts Configuration ****).

In the pluging system there are two funtions (CanConfig, Configure) in order to activate and use de 'Configure Selected pluging' buttom (the "Tool" buttom, at right of the 'Import name', between 'Launch Import' buttom and 'Set Overview Options') (see Programming own plugins (http://www.videodb.info/forum_en/index.php/topic,764.0.html)).

Somebody know how do that for the Scripts?
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: fmb64 on September 16, 2018, 03:34:46 pm
Quote
Is it possible to hide the flickering windows or run them minimized?
unfortunately not because the PVD makes the launch. The MOD tries to hide it and recover the focus of windows PVD, but it can not be done more.
What about (maybe optionally) not recovering the focus of the PVD window, maybe one can work with another software without the flickering and even not jumping out of a full screen game. ;)
Quote
Have you tried the TheMovieDB script? This free provider is specially designed for this type of programs: download only the precise information in text mode, so it works very fast.
Ok, this seems to be an nice alternative.

It imports a screenshot of the movie, how to disable this?

Please add a german version of the script - I copied the english one and modified it, works  :)
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on September 18, 2018, 08:34:46 am
Quote
Please add a german version of the script - I copied the english one and modified it, works  :)
Great new to have a German user.
It's true that TMDB have a german version but I don't know german. But don't worry I'd prepared the scripts for easy translation: I use a txt file with the first line in english and the second in the 'foreign' language.
I'd done the draft of german translation with Google attached. May you correct those files in order to incorporate to the MOD version?
BTW, if there is other 'foreign' person with other language need all help are welcome.

Quote
It imports a screenshot of the movie, how to disable this?
There are two way to control the saved of information in the PVD database.
1) The PVD standard with the 'Set Overwrite Options for the selected plugin' (the "Calendar" button, at right of the script 'Import name').
2) The custom fields and others aren't included in this option so the scripts have hardcode parameters that now are easily changed launching the first script '**** Scripts Configuration ****' (you must have selected one movie or one person).
This is the way to disable the screenshot save:
a) Launch the PVD MOD Configuration Script:

(http://www.videodb.info/forum_en/index.php?action=dlattach;topic=4134.0;attach=6686)

b) With this the PDV MOD Script Configuration windows opens. In the wanted script sheet, Enable or disable your preference (Screenshot with a horizontal blue arrow)

(http://www.videodb.info/forum_en/index.php?action=dlattach;topic=4134.0;attach=6688)

c) Saved it (vertical blue arrow)

BTW you can activate or deactivate a script checking the Active box (green circle)

Quote
What about (maybe optionally) not recovering the focus of the PVD window, ...
This isn't easy because PVD interacts with the focus but I have understood your request: To work with PVD in second plane completely.
With this information I'll try to do something if possible.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: fmb64 on September 18, 2018, 02:53:36 pm
That’s very cool to interact with the programmer to help improving the software. :)

Quote
Great new to have a German user.

Yes, in fact Swiss user but no matter  ;)

I corrected the german translation, it is a bit difficult because I think the English is not perfect, too   ;) ;)

Quote
There are two way to control the saved of information in the PVD database.

Thank you very much, now I got it how to configure the script.

The launch of the configuration script does not work if PVD is stored in a path containing spaces (for example "D:\Personal Video Database")   see attached error. (how to embed the screenshot?)

On "Save Config" PVD restarts with the sample movie database instead of the last I had open.

BTW, the duration of the movie is imported to 'Features' instead of 'Duration'.

Quote
This isn't easy because PVD interacts with the focus but I have understood your request: To work with PVD in second plane completely.
With this information I'll try to do something if possible.

That would be awesome, because I still prefer iMDb (a little bit)   :)
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 18, 2018, 03:24:41 pm
BTW, the duration of the movie is imported to 'Features' instead of 'Duration'.

Yes, this is a mistake that will be corrected in the next IMDB_ [EN] [HTTPS] script version. Within a few days, the patches will be posted from my site (I need to do some tests because I've fixed the errors) in this topic and then the new IMDB_ [EN] [HTTPS] script version will be published.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: fmb64 on September 18, 2018, 04:11:56 pm
Yes, this is a mistake that will be corrected in the next IMDB_ [EN] [HTTPS] script version.

Thanks guys, I really appreciate your work for keeping this software alive.  :)
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 18, 2018, 04:36:19 pm
IMDB_ [EN] [HTTPS] script

Quote
//Download methods-------------------------------------------------------------------------------------------------------
  dmGET           = 0;
  dmPOST          = 1;
//Movie fields----------------------------------------------------------------------------------------------------------
  mfURL         = 0;
  mfTitle      = 1;
  mfOrigTitle   = 2;
  mfAka         = 3;
  mfYear         = 4;
  mfGenre      = 5;
  mfCategory      = 6;
  mfCountry      = 7;
  mfStudio      = 8;
  mfMPAA         = 9;
  mfRating      = 10;   //This is 'Additional rating', not 'Rating'
  mfTags         = 11;
  mfTagline      = 12;
  mfDescription   = 13;
  mfDuration      = 14;
  mfFeatures      = 15;

//Credits  types----------------------------------------------------------------------------------------------------------

This part of the code is required because otherwise the movie duration will not be added to the database. This part of the code is needed because of these two parts of the code in the script.

The first is for the following part:


Quote
    //Get the original "Runtime" (in minutes) as ~length~ instead of the duration of the user movie copy
    If Not(PRESERVE_USER_MOVIE_CHARS) Then Begin
        curPos:=Pos('<h4 class="inline">Runtime:</h4>',HTML);                                      //WEB_SPECIFIC.
        If 0<curPos Then Begin 
            ItemValue:=TextBetWeen(HTML,'M">',' min</time>',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
            //ItemList:=ItemList+'Original Runtime: '+ItemValue+' min.<br>';
            //LogMessage('      Get result Original Runtime_1:'+ItemValue+'||');
            //AddFieldValueXML('length',ItemValue);
            AddFieldValue(mfDuration,ItemValue);      
            LogMessage('      Get result lenght:'+ItemValue+'||');
        End;
    End;

The second is for Function ParsePage_IMDBMovieTECHNICAL:

Quote
Function ParsePage_IMDBMovieTECHNICAL(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit
    //Retrieve: ~features~
  Var
    //curPos:Integer;
    curPos,endPos:Integer;
    ItemValue,ItemList:String;     
  Begin
    LogMessage('Function ParsePage_IMDBMovieTECHNICAL BEGIN=====================||');
    Result:=prFinished;  //It will change to prError if any big problem with exit; 
    //Get "Technical Specifications" info
    curPos:=Pos('<h1 class="header">Technical Specifications</h1>',HTML);
    if (curPos=0) then Exit;
    EndPos:=curPos; 
    //Get "Runtime" info
    curPos:=Pos('<td class="label"> Runtime </td>',HTML);                                      //WEB_SPECIFIC.
   If 0<curPos Then Begin 
      ItemValue:=TextBetWeen(HTML,' (',' min)',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
        LogMessage('      Get result lenght from Runtime:'+ItemValue+'||');
        If ItemValue <> '' then begin       
    //Get the original "Runtime" (in minutes) as ~length~ instead of the duration of the user movie copy
            If Not(PRESERVE_USER_MOVIE_CHARS) Then Begin 
                AddFieldValue(mfDuration,ItemValue);
                //AddFieldValueXML('length',ItemValue);
                LogMessage('      Get result lenght from Original Runtime:'+ItemValue+'||');
            End;
            //ItemList:=ItemList+'Runtime_1'+'<br>'+ItemValue+'<br><br>';
        End;
    End;                 
    //Get "Runtime" info
    ItemValue:=HTMLValues2(HTML,'Runtime','</tr>','<td>','</tr>','<br><br>',EndPos);
    ItemValue:=StringReplace(ItemValue,'                                ','<br>',True,False,True);
    ItemValue:=StringReplace(ItemValue,'                      ','<br>',True,False,True);
    ItemValue:=StringReplace(ItemValue,'          ','',True,False,True);
    If ItemValue <> '' then LogMessage('      Get result Runtime:'+ItemValue+'||');
    if ItemValue <> '' then ItemList:=ItemList+'Runtime'+'<br>'+ItemValue+'<br><br>';   
       
    //Get "Sound Mix" info

Here it is necessary to add a part of the code for  // Get "Runtime" info as I added it here because otherwise it will not correctly transfer the information for "Runtime".
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 18, 2018, 05:56:46 pm
IMDB_ [EN] [HTTPS] script

A minor correction for ParsePage_IMDBSearchTitle code, where repeatedly did not show all hits when searching for results or did not show the year when movie release.


Quote
Function ParsePage_IMDBSearchTitle(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prDownload; (Global var DownloadURL=IMDB page) if there is one results.
    //     Result:=prList; if there are several results. (Don't work with Preferences/Plugings/Silent Mode).
    //     Result:=prError; If not results
  Var
      curPos,ResultsNumber:Integer;
     endPos:Integer;
      ItemValue,ImageFile:String;
      Title,Year,MovieURL,PreviewURL:String;
  Begin
.
.
.
            //Get MovieURL  (Always must it has)
            MovieURL:=TextBetWeen(HTML,'<a href="/title/','/',false,curPos);               //Strings which opens/closes the data. WEB_SPECIFIC
            MovieURL:= BASE_URL_PRE+MovieURL+BASE_URL_SUF;
            LogMessage('      Parse Results URL:'+MovieURL+'||');
            //If true HTPPS link PVdB gives "Socket Error # 11001"
            //If ancient http, get redirected and give "IOHandler value is not valid
            //If invalid url, "HTTP/1.1 400 Bad Request"               
            MovieURL:=CHEAT_PREFIX_URL+MovieURL;                 //'Cheat' URL for avoid HTTPS issue in GET return.
            LogMessage('      Parse Results CHEAT URL:'+MovieURL+'||');
            //Get Title (Always must it has)
            Title:=TextBetWeen(HTML,'>','<',false,curPos);                               //Strings which opens/closes the data. WEB_SPECIFIC
            LogMessage('      Parse Results Title:'+Title+'||');
            //Get Year
            //Year:=TextBetWeen(HTML,'(',')',false,curPos);                               //Strings which opens/closes the data. WEB_SPECIFIC
            endPos:=PosFrom('</span>',HTML,curPos);
            curPos:=endPos-5;
            Year:=IntToStr(StrToInt(Copy(HTML,curPos,4)));

            LogMessage('      Parse Results in Year:'+Year+'||');
            AddSearchResult(Title,'',Year,MovieURL,PreviewURL);
            //Just to check the website (Only to check the web page) not PVdB valid result.
            if CHECK_WEBSITE then AddSearchResult('                             '+#8729+' Just to check the website: '+StringReplace(MovieURL,CHEAT_PREFIX_URL,'',True,False,False),'','',StringReplace(MovieURL,CHEAT_PREFIX_URL,'',True,False,False),'');
            curPos:=PosFrom('<div class="lister-item mode-simple">',HTML,curPos)          //String which opens the Web Result item List data. WEB_SPECIFIC
        End;

For ~ Metascore: ~ information must be all part of the code because the web pages share different parts of the source code for Metascore data.

Quote
    //~Metascore:~
    curPos := PosFrom('> <div class="metacriticScore score_favorable titleReviewBarSubItem">',HTML,EndPos);
    if curPos>0 then begin
        curPos:=curPos+Length('> <div class="metacriticScore score_favorable titleReviewBarSubItem">');      
        EndPos:=PosFrom('</span>',HTML,curPos);
        ItemValue:=RemoveTags(Copy(HTML,curPos,endPos-curPos+2),false);
        ItemValue:=StringReplace(ItemValue,'0</',',0',True,False,True);
        ItemValue:=StringReplace(ItemValue,'1</',',1',True,False,True);
        ItemValue:=StringReplace(ItemValue,'2</',',2',True,False,True);
        ItemValue:=StringReplace(ItemValue,'3</',',3',True,False,True);
        ItemValue:=StringReplace(ItemValue,'4</',',4',True,False,True);
        ItemValue:=StringReplace(ItemValue,'5</',',5',True,False,True);
        ItemValue:=StringReplace(ItemValue,'6</',',6',True,False,True);
        ItemValue:=StringReplace(ItemValue,'7</',',7',True,False,True);
        ItemValue:=StringReplace(ItemValue,'8</',',8',True,False,True);
        ItemValue:=StringReplace(ItemValue,'9</',',9',True,False,True);
        AddCustomFieldValueByName('Metascore',FloatToStr((StrToFloat(ItemValue) * 1)));
        LogMessage('      Get result custom field Metascore_0:'+ItemValue+'||');
    End;   

    //Get ~Metascore~
    curPos:=PosFrom('> <div class="metacriticScore score_unfavorable titleReviewBarSubItem">',HTML,EndPos);
    if curPos>0 then begin
        curPos:=curPos+Length('> <div class="metacriticScore score_unfavorable titleReviewBarSubItem">');
        EndPos:=PosFrom('</span>',HTML,curPos);
        ItemValue:=RemoveTags(Copy(HTML, curPos,endPos-curPos+2),false);
        ItemValue:=StringReplace(ItemValue,'0</',',0',True,False,True);
        ItemValue:=StringReplace(ItemValue,'1</',',1',True,False,True);
        ItemValue:=StringReplace(ItemValue,'2</',',2',True,False,True);
        ItemValue:=StringReplace(ItemValue,'3</',',3',True,False,True);
        ItemValue:=StringReplace(ItemValue,'4</',',4',True,False,True);
        ItemValue:=StringReplace(ItemValue,'5</',',5',True,False,True);
        ItemValue:=StringReplace(ItemValue,'6</',',6',True,False,True);
        ItemValue:=StringReplace(ItemValue,'7</',',7',True,False,True);
        ItemValue:=StringReplace(ItemValue,'8</',',8',True,False,True);
        ItemValue:=StringReplace(ItemValue,'9</',',9',True,False,True);
        AddCustomFieldValueByName('Metascore',FloatToStr((StrToFloat(ItemValue) * 1)));
        LogMessage('      Get result custom field Metascore:'+ItemValue+'||');
    End;
    //~Metascore:~
    curPos := PosFrom('> <div class="metacriticScore score_mixed titleReviewBarSubItem">',HTML,EndPos);
    if curPos>0 then begin
        curPos:=curPos+Length('> <div class="metacriticScore score_mixed titleReviewBarSubItem">');      
        EndPos:=PosFrom('</span>',HTML,curPos);
        ItemValue:=RemoveTags(Copy(HTML,curPos,endPos-curPos+2),false);
        ItemValue:=StringReplace(ItemValue,'0</',',0',True,False,True);
        ItemValue:=StringReplace(ItemValue,'1</',',1',True,False,True);
        ItemValue:=StringReplace(ItemValue,'2</',',2',True,False,True);
        ItemValue:=StringReplace(ItemValue,'3</',',3',True,False,True);
        ItemValue:=StringReplace(ItemValue,'4</',',4',True,False,True);
        ItemValue:=StringReplace(ItemValue,'5</',',5',True,False,True);
        ItemValue:=StringReplace(ItemValue,'6</',',6',True,False,True);
        ItemValue:=StringReplace(ItemValue,'7</',',7',True,False,True);
        ItemValue:=StringReplace(ItemValue,'8</',',8',True,False,True);
        ItemValue:=StringReplace(ItemValue,'9</',',9',True,False,True);
        AddCustomFieldValueByName('Metascore',FloatToStr((StrToFloat(ItemValue) * 1)));
        LogMessage('      Get result custom field Metascore_1:'+ItemValue+'||');
    End;
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 18, 2018, 06:20:23 pm
IMDB_ [EN] [HTTPS] script

Full Cast and Crew section

For directors, you also need to add a snippet of code in order for the data to be downloaded correctly.


Quote
    //Go to "Directed by" ~crew~ctDirectors
    curPos:=Pos('<h4 class="dataHeaderWithBorder">Directed by',HTML);                     //Strings start which opens the block content data. WEB_SPECIFIC   
   If 0<curPos Then Begin
      curPos:=PosFrom('</h4>',HTML,curPos);                                              //Strings end which opens the block content data.  WEB_SPECIFIC
      curPos:=curPos+Length('</h4>');                                                    //Strings end which opens the block content data.  WEB_SPECIFIC
      //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~ n the ~crew~
      endPos:=PosFrom('</table>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
      ItemList:=Copy(HTML,curPos,endPos-curPos);
      //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');
      curPos:=Pos('<td class="name">',ItemList)                                         //String which opens the subList data. WEB_SPECIFIC
      index:=1;
      While curPos>0 Do Begin
         If (index>PEOPLE_LIMIT) Then break;     //Limited depassed (Remember index begin in 0).
         //Get PersonURL (Always must it has)
         PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF;     //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
         //Get Name (Always must it has)
         Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC      
         LogMessage('      Parse Results Name:'+Name+'||');
         //Get Role: PVD don't save Role in crew people but is added for Ivek23 as LogMessage compatibility
         Role:=TextBetWeen(ItemList,'<td class="credit">','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results Role:'+Role+'||');
         AddMoviePerson(Name,'','',PersonURL,ctDirectors);
         LogMessage('      Get results Director:#'+IntToStr(index)+'|'+Name+'|'+PersonURL+'||ctDirectors'); 
         curPos:=PosFrom('<td class="name">',ItemList,curPos)                                            //String which opens the subList data. WEB_SPECIFIC
         index:=index+1;
      End;
    End;

For Function ParsePage_IMDBMovieMPAA is also a small correction.

Quote
Function ParsePage_IMDBMovieMPAA(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError;  If any big problem? with exit
    //Retrieve: ~MPAA~
  Var
    curPos:Integer;
    mpaaValue:String;
  Begin
.
.
.
   mpaaValue:=StringReplace(mpaaValue,'update: ','',False,False,True);                         //Cleanig value. WEB_SPECIFIC
    curPos:=Pos('Edit',mpaaValue);                               //WEB_SPECIFIC.
    If 0<curPos then mpaaValue:=Copy(mpaaValue,0,curPos-1);   

    If (mpaaValue='Approved') Then mpaaValue:='Rated G';
.
.
.
    AddFieldValueXML('mpaa',mpaaValue);
    LogMessage('      Get result MPAA='+mpaaValue+'||');
    LogMessage('Function ParsePage_IMDBMovieMPAA END=====================||');
  End; //BlockClose

IMDB_ [EN] [HTTPS] _00 script is attached.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 18, 2018, 06:28:33 pm
Yes, this is a mistake that will be corrected in the next IMDB_ [EN] [HTTPS] script version.

Thanks guys, I really appreciate your work for keeping this software alive.  :)

Welcome.

I tried and posted corrections. If I find anything else, I'll post it later.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on September 18, 2018, 08:12:10 pm
Script Movie Configuration bug for http://vvveasy.altervista.org/wp-content/uploads/2018/08/PVD_0.9.9.21_MOD-V.2018.08.08.zip
The launch of the configuration script does not work if PVD is stored in a path containing spaces (for example "D:\Personal Video Database")   see attached error.

Thanks both. I have not understood the first time. Corrected in the attached patch that oame with the TheMovie [DE] scripts too.
NOT YET the Ivek23 alerts corrected in this patch.

Two questions to Ivek23 about duration info.
1) What is the diference between  "AddFieldValueXML('length',ItemValue);" and "AddFieldValue(mfDuration,ItemValue); " with "mfDuration = 14;"
2) PVD have a command "Re-read file information" in order to fill the next fields:
(http://www.videodb.info/forum_en/index.php?action=dlattach;topic=4134.0;attach=6696)
Is this Duration yours field? In your PVD version is not it ~length~?
a) If yes, I have added a new script parameter in order to preserve it for the user with a movie file copy
     PRESERVE_USER_MOVIE_CHARS = True ; //Preserve the user movie copy characteristics (length and languages channels) instead of the original runtime and languages.
b) If not What PVD field is? I don't be able to find it in my PVD database or in the forum documentation and it overwrites the .

Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 19, 2018, 01:57:41 pm
Two questions to Ivek23 about duration info.
1) What is the diference between  "AddFieldValueXML('length',ItemValue);" and "AddFieldValue(mfDuration,ItemValue); " with "mfDuration = 14;"

Otherwise, this means the same and the same normal field in the PVD database, but in IMDB_ [EN] [HTTPS] script example "AddFieldValueXML (" length ", ItemValue);" does not transfer any length or duration of a movie. It's completely different with "AddFieldValue (mfDuration, ItemValue);" which transfers the length or duration of a movie website, so it's necessary "mfDuration = 14;" he also needs it.

If already IMDB_ [EN] [HTTPS] script is being edited to work, it is also right that this part of the code is also edited in order to work properly. This is about certain PVD users, who use multiple databases in order to choose the information they want.

My example of how I use the length or duration of a movie in the database:

I am using one database with information about a specific movie such as length and
similar to the movies folder stored in the archive on external disks. In the rest of the database, for example, I have, for example, saved only the length of the movie, which information is downloaded from the web site using plug-ins or scripts.

2) PVD have a command "Re-read file information" in order to fill the next fields:
(http://www.videodb.info/forum_en/index.php?action=dlattach;topic=4134.0;attach=6696)
Is this Duration yours field? In your PVD version is not it ~length~?
a) If yes, I have added a new script parameter in order to preserve it for the user with a movie file copy
     PRESERVE_USER_MOVIE_CHARS = True ; //Preserve the user movie copy characteristics (length and languages channels) instead of the original runtime and languages.

I've already answered partially. Of course, all this is stated in my database and is true, but not for everyone.

PRESERVE_USER_MOVIE_CHARS = True; keeps the length of the film.

PRESERVE_USER_MOVIE_CHARS = True; however, it is not true for the original language because this is a particularly normal field in the database. The Original Language field always downloaded the information for the original language from the website (with IMDB only the first written language, which is very much often English language). There is only a language box, but it does not have a direct link to the original language field.

b) If not What PVD field is? I don't be able to find it in my PVD database or in the forum documentation and it overwrites the .

"length" is the XML mark for the duration field and is in the skins (find it there), the Duration record is for a length - "length" field and it is visible in the PVD database.

Recommendation:

This option  PRESERVE_USER_MOVIE_CHARS = True;  it would be necessary to define for the time and separately for the language, so that each user can decide on his own data for which information he will keep and what information he will replace with other data.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 19, 2018, 03:38:32 pm
IMDB_ [EN] [HTTPS] script

// Get ~ imdbrating ~, ~ IMDB_Votes ~ code has been changed again and now the code again works as it worked on 2018.08.08, when the last PVD_1.0.2.7_MOD-V.2018.08.08 was created.

Quote
    //Get ~imdbrating~, ~IMDB_Votes~
    curPos:=Pos('<span itemprop="ratingValue">',HTML);                      //WEB_SPECIFIC.
    If 0<curPos Then Begin                         
       ItemValue:=TextBetWeen(HTML,'<span itemprop="ratingValue">','<',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=StringReplace(ItemValue,',','.',True,True,False);                           //Decimal comma spanish separator to point english separator.
       AddFieldValueXML('imdbrating',ItemValue);
       LogMessage('      Get result imdbrating:'+ItemValue+'||');
       ItemValue:=TextBetWeen(HTML,'<span class="small" itemprop="ratingCount">','</span>',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=StringReplace(ItemValue,'.','',True,True,False);            //Remove thousand point spanish separator.
       ItemValue:=StringReplace(ItemValue,',','',True,True,False);            //Remove thousand comma english separator.
       AddCustomFieldValueByName('IMDB_Votes',ItemValue);
       LogMessage('      Get result IMDB_Votes:'+ItemValue+'||');
    End;

Fixed IMDB_ [EN] [HTTPS] _00 script is attached.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on September 19, 2018, 06:25:36 pm
Attached the Acumulative Scripts Patch date:2018.09.19 (to copy in PVD folder for overwriting the existing script files).
All Ivek23 alerts just to this messages included but:

Please, Ivek test the ~length~ question.

With ~length~ it works for me in the two PVD versions but if you don't have movie file you must edit the movie panel in order to see it (add a field by number is [deprecated] in PVD manual).

Note:  I have removed the PRESERVE_USER_MOVIE_CHARS parameter because this can be done with the 'Set Overwrite Options for the selected plugin' (the "Calendar" button, at right of the script 'Import name' in the 1.0.2.7 version and in 'Preferences/Plugin Preferences' in the 0.9.9.21).
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on September 19, 2018, 07:00:30 pm
I'd like to request that the AdultDVDEmpire script be adapted for this mod. I've been sort of maintaining the ADE script in the past, but I can lo longer get PVD and Proxomitron to play nice together. From what I can glean from looking at the mod's scripting, the mod uses an external program to download all https:// web pages, then parses them locally. This is beyond my capabilities to adapt.
The script development has changed a little but is not far from the traditional aproach and all funtional code can be re-used ... and I can help you.

For test if your ancient script work you can follow those steps:
1) In Explorer/Firefox browse the page of one movie in your provider and "Save as" / "Web page, only HTML"  it in a text file "Example_movie.htm" in your Scripts folder
2) In your script:
     a) In the function GetDownloadURL, code the result as "Result:='http://httpbin.org/response-headers?key=Null';" (this avoid the HTTPS PVD problem)
     b) In the funtion ParsePage, recover the dowloaded provider page with "HTML:=FileToString(GetAppPath+'Scripts\'+'Example_movie.htm');"
3) After this instruction you can process the page with the script in order to get all the information.
4) When you arrive a good result for this unique movie, publish the script in the forum that I pass the code to the new download script process and then you can try it with multiple and real pages and finish the script.
Note: For process the results of the search page you can use the same method.

Let's go, you can do it!  ;)   ... and all people are invited too!  ;D
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 20, 2018, 07:13:30 am
Please, Ivek test the ~length~ question.

~ length ~ works but this part of the code AddFieldValueXML ('length', ItemValue); it prints Lenght number 00:01:37, which is wrong.

This part of the AddFieldValue code (14, ItemValue); // Filed number 14 = Duration. it prints Lenght number 01:37:00, which is absolutely correct.

With ~length~ it works for me in the two PVD versions but if you don't have movie file you must edit the movie panel in order to see it (add a field by number is [deprecated] in PVD manual).

I kept the second option in the enclosed IMDB_ [EN] [HTTPS] script where I also corrected the entire Function ParsePage_IMDBMovieTECHNICAL and the beauty of // Go to "Directed by" ~ crew ~ ctDirectors in Full Cast and Crew section.

Script is attached.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Miguelh1020 on September 20, 2018, 02:35:05 pm
Hi guys! Thanks for keeping the dream alive!
I noticed a difference with my olf file made with the old PVD:
The list of TAGS now appears as CATEGORY, and they are included in different lists.
Also I can't seem to update the "awards" section on the people database.
Also also, I used to add manually the "Duration" of the film, now it loads automatically, but it's in the "Features" section.

These are all minor details. The software is functional now  :) :) :)

Oh I noticed something else, It would seem like TheMovieDB doesn't upload the "complete" cast. Just the main characters
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 20, 2018, 03:35:52 pm
Hi guys! Thanks for keeping the dream alive!
I noticed a difference with my olf file made with the old PVD:
The list of TAGS now appears as CATEGORY, and they are included in different lists.
Also I can't seem to update the "awards" section on the people database.
Also also, I used to add manually the "Duration" of the film, now it loads automatically, but it's in the "Features" section.

These are all minor details. The software is functional now  :) :) :)

Now it should be (at least I hope it will be) again as it was several months ago or years ago, at least some of the things we did to make some corrections in this direction.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on September 20, 2018, 07:46:48 pm
Attached the Acumulative Scripts Patch date:2018.09.20 (to copy in PVD folder for overwriting the existing script files).
All alerts just to this message applied.


Quote
~ length ~ works but this part of the code AddFieldValueXML ('length', ItemValue); it prints Lenght number 00:01:37, which is wrong.
This part of the AddFieldValue code (14, ItemValue); // Filed number 14 = Duration. it prints Lenght number 01:37:00, which is absolutely correct.

Wonderfull, Ivek23, you have discover the bug in PVD= Field number 14 works in minutes and ~length~ field works in seconds but two are Duration. Easy to solve multiplying by 60.

Quote
I noticed a difference with my olf file made with the old PVD

Thank you for your alert, Miguelh1020, try to apply this acumulative Patch over "PVD MOD 2018.08.08" Scripts folder.

Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 21, 2018, 01:18:13 pm
IMDB_ [EN] [HTTPS] script is currently working properly. I'm testing and I'm looking for possible errors, if they still exist.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: bobwal on September 22, 2018, 01:50:24 am
hello, have been a long time user of this wonderful program , but i am completely lost as to how to use this mod. is there a clear ,  step by step explanation of how to install this mod over the old one. i have tried to do it with the instructions in the early part of this discussion but it did not work. i am not a programmer so a lot of what you say is very easily misunderstood . i really hope some one can help.

would love to get this program working the way it used to.

many thanks
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 22, 2018, 08:37:04 am
hello, have been a long time user of this wonderful program , but i am completely lost as to how to use this mod. is there a clear ,  step by step explanation of how to install this mod over the old one. i have tried to do it with the instructions in the early part of this discussion but it did not work. i am not a programmer so a lot of what you say is very easily misunderstood . i really hope some one can help.

would love to get this program working the way it used to.

many thanks

Welcome to the forum, bobwal.

Leave the old PVD version as it is, do not need to overwrite it.

Download PVD_1.0.2.7_MOD-V.2018.08.08 on the link below.
http://vvveasy.altervista.org/wp-content/uploads/2018/08/PVD_1.0.2.7_MOD-V.2018.08.08.zip
Download PVD_1.0.2.7_MOD-V.2018.09.22 on the link below.
http://vvveasy.altervista.org/wp-content/uploads/2018/09/PVD_1.0.2.7_MOD-V20180922.zip


Since PVD_1.0.2.7_MOD-V.2018.08.08 is modified in a portable version, you unpack it. Previously, create a folder named Portables (Portable) in the C: \ Users \ Public or other partition folder with that name. Then copy the unpacked folder named PersonalVideoDB into the previously created Portables (Portable) folder. In the Portables (Portable) folder you also copy your movie PVD database, as this will preserve the originasl database. This mode will change this database and if you accidentally use the old 0.9.9.21 version, then you will no longer be able to use this modified database with this old version.

You can also copy your skins to the Skins folder for Movies in the Movies folder and People in the People folder.

Now, run the portable.bat file in the PersonalVideoDB folder and it will have to work. But if it does not work yet, let us know and we will try to help you otherwise. Starting a portable.bat file will not lose any old data, because all the settings will be stored in the pvdconf.bak file that is in this folder, or it will be created when you run the portable.bat file.

I hope this will be written to you for good help and it will work as you wish.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on September 22, 2018, 10:44:56 am
hello, have been a long time user of this wonderful program , but i am completely lost as to how to use this mod. is there a clear ,  step by step explanation of how to install this mod over the old one. i have tried to do it with the instructions in the early part of this discussion but it did not work. i am not a programmer so a lot of what you say is very easily misunderstood . i really hope some one can help.
would love to get this program working the way it used to.
many thanks
I have rewritten the post 'Passing from original PVD version' (http://www.videodb.info/forum_en/index.php/topic,4134.msg20627.html#msg20627) in order to help to this situation.
I understand that you have installed the MOD version and you have problems to recover your movie database over the MOD version.
If you are a ancient PVD 1.0.2.7 user, the easiest way to change to the MOD version is Copy and Paste the movies.
If you are a ancient PVD 0.9.9.21 user, follow the Ivek23 post steps. Or, other way longer, pass first to the original 1.0.2.7 and then Copy and Paste the collection.

Note: If you are a normal user, I strongly recommends use the SyMenu (https://www.ugmfree.it/) free launcher to install and keep updated the portable Personal Video Database 1.0.2.7 MOD version (after pass your ancient collection). It's easy and completely free without charges, advertisements, etc. (same philosophy as PVD) and perhaps you discover other advantages (for instance the Win10 users don't need use the infamous Windows Tiles).
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: bobwal on September 23, 2018, 06:35:29 pm
thank you for your replies .

my version that i am using is 1.0.2.7 (old one)

i tried what you said to do but it did not work . i tried to copy over files from my old version to the portable version but it kept telling me i have duplicate files , so i cleared and deleted the files in the portable version but to no avail. i dont know what i did wrong.

i set up a folder called portables in public as told and then put the latest version of 1.0.2.7 mod into portables folder and extracted , i then ran the bat file and the portable version started up . so now i have 2 versions running and started to copy/paste my files and then the problems started with duplicates in the portable mod version

did i do the right thing?

again thanks for the help
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on September 23, 2018, 06:47:52 pm
Run first version in the PersonalVideoDB folder. Run the second version in the renamed PersonalVideoDB1 folder. Now I hope that there should be no problems.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on September 23, 2018, 08:13:39 pm
and started to copy/paste my files and then the problems started with duplicates in the portable mod version
So I think that you have choose the way 'Copy & Paste' movies in PVD, isn't it?
When you say 'my files' what do you want to say? Because in this way you must copy and paste de PVD records, the 'movie lines': Amadeus, Casablanca, etc from the database of the old version in the MOD PVD.
Have you tried first one by one movie in order to learn?
Can you attach one error window screenshot?
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: afrocuban on September 26, 2018, 10:42:40 pm
Dear VVV and Ivek,

Thank you so much for your hard work and for your devotion to PVD.

I do thoroughly test PVD-MOD and I have found it almost perfect.
Nevertheless, I do have a problem while importing online data from IMDb. The problem can be seen in the screenshot.


It happens only during update of some movies, not for all. It doesn't happen when enter new records/movies.

Any help is more than welcome!
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on September 27, 2018, 06:04:00 pm
Nice to know about you, afrocubam  :D
You have got a hard error, it seems that PVD tries to dulicate some record with a SQL instruction.
Some information in in this link: Why am i getting a primary/unique key violation/ (https://sqlstudies.com/2014/07/23/why-am-i-getting-a-primaryunique-key-violation/)

Some test in order to detects/reproduce the problem:

Really is a strange error,  ???
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: afrocuban on September 28, 2018, 12:37:52 am
Nice to know about you, afrocuban  :D
Thank you! I really enjoy it!
Quote
You have got a hard error, it seems that PVD tries to dulicate some record with a SQL instruction.
Some information in in this link: Why am i getting a primary/unique key violation/ (https://sqlstudies.com/2014/07/23/why-am-i-getting-a-primaryunique-key-violation/)
Thank you, dear VVV. I read and I think it's exactly the problem I experienced. Unfortunately, I do not programming, so it wasn't too much helpful for me, but I understand what it says.


Quote
Some test in order to detects/reproduce the problem:
  • Have you more Customs Fields that the MOD estandar (http://www.videodb.info/forum_en/index.php/topic,4134.msg20627.html#msg20627)? (In the last version I have include 'Updated').
Yes, I do. A lot more.

Quote
  • The error cames in the same movies? If yes, can tell us.
YES!

Quote
  • The error cames with other scripts/provider? If yes, can tell us.
No. Checked with FA and TMDb.

Quote
  • Can you try disabling IMDB pages donwloading with the ***Script M. Configuration***? First try only dowloading the base page, after adding the Certification page, after adding the  "Also Know as' page,... I suspect of the retreive of people because it is the part less depurated in PVD and the one that handles more links.
I must admit I don't know how to use ***Script M. Configuration***. When I call it, only dialog box with options for all scripts pops up, and when i save configuration, PVD restarts.

I try only to update Title, Top_250 and IMDb_Votes (Yes, you made a problem for me, hahaha. My custom fields were "Top 250:" and "IMDB Votes:", so now because you gave them different names, I have to update 7000 movies, and I ran into this problem when updating, hahah").

But, I did what you suggest - I disabled updating fields in IMDb script and I discovered that if I disable updating some or all fields, everything works, for now!? Not all of these fields cause problems for movies. For some movie only one field causes problem, for other movie other two fields, and for some movies all these fields. Sometimes, If I repeat import, it succeeds??? Strange thing is that I discovered also that for some movies I cannot manually  delete Title when edit movie. If I delete it, when I press to exit edit mode, deleted Title comes back? Also the same for Description for some some movies - I delete it and it comes back when I exit edit mode?!

Quote
Really is a strange error,  ???
Definitely.

Conclusion: It has problems with overwriting existing fields, that cannot be deleted even manually


At the end: a great respect and thank you for "Metascore" field!

Kind regards
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on September 29, 2018, 09:44:42 am
Quote
Have you more Customs Fields that the MOD estandar? ... A lot more.
The PVD is very stable but it have some inner problems: Be careful with spaces in the Custom Fields.
Quote
The error cames in the same movies? ... YES!
If you tell us the movie title perhaps Ivek32 or myself can reproduce the problem.
Quote
The error cames with other scripts/provider? ...  No. Checked with FA and TMDb.
So the problem is with the IMDB scritp. This is a very complicated script. In the default version it dowloads 6 web pages, everyone with his own routine. So is very important to know what page 'fails'. BTW, if you are updating only some fields you are downloading a lot of not needed pages.
Quote
I must admit I don't know how to use ***Script M. Configuration***.
I must rebuke you  ;D.
You are losting a lot of time. I rewrite the ***Script M. Configuration*** help in this post Controling the retreived information in PVD MOD (http://www.videodb.info/forum_en/index.php/topic,4134.msg20627.html#msg20627)
If you are trying only to update Title, Top_250 and IMDb_Votes You only need the principal page so you must uncheck the other 5 pages! (7000 movies x 5 pages = 35.000 downloads saved) Please, make a little efort  ;)
Quote
My custom fields were "Top 250:" and "IMDB Votes:", so now because you gave them different names, I have to update 7000 movies,
You don't need create a new fields, you can rename its. See this post (http://www.videodb.info/forum_en/index.php/topic,4134.msg20672.html#msg20672)
Quote
I disabled updating fields in IMDb script and I discovered that if I disable updating some or all fields, everything works, for now!? Not all of these fields cause problems for movies. For some movie only one field causes problem, for other movie other two fields, and for some movies all these fields.
Please, tell us those fields and movies! As I wrote in the forum, there is some uses of the database for the script not very safe (use Description and Comments, fields that are used for movie information and user file information, etc). The PVD database standar is not completely finished.
One more example of this, if you wants to update the Comments or the user Rating (not the IMDBrating) with a script you must clear it before (normaly with the Mass Editor)
Quote
If I delete it, when I press to exit edit mode, deleted Title comes back? Also the same for Description for some some movies - I delete it and it comes back when I exit edit mode?!
Perhaps you have people information (awards, featured, etc) with the same movie that you have.
PVD have information of the movie by two way: By Movie list and by People info (for this I not download the People's awards, it brings to PVD a lot of unknowed movies).

 
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: afrocuban on October 01, 2018, 11:54:36 pm
Dear VVV,

Thank you so much for your feedback!
I'll try to be brief as possible in my response.

1. I have never had any problems with spaces, although aware of possible problems. I tested them before start to use them almost 9 years ago.
2. To reproduce the problem, maybe better idea is to make a new database that consists only a few problematic movies and then send it to you (with my pvdconf.ini and my skin)? Anyway, some movies are:
 imdb.com/title/tt0034583/
 imdb.com/title/tt0082176/
 imdb.com/title/tt0070518/
 imdb.com/title/tt0074252/
3. Probably the problem is with IMDb script. But, not only movie, but now I suspect that people are problem too, even before I started to use PVD MOD. I mean, and I'm not sure at all, when we started to use IMDb scripts I noticed that I have duplicate entries for people. It's like some people were imported by nostra's imdb.dll and their duplicates with scripts. So now, for example I have two (sometimes, even more) actors with the same name in database and their filmography is split (to movies imported by imdb.dll and other part with script). If I update both, I get full filmography in both, but links to movies are still split. If I delete any name, I loose them in movies linked to that name. That is why I suggested to send you a small database. Is it even possible to overcome this?
4. I know you won't believe me, but renaming custom fields never worked for me. Of course I tried it once again before started to update database this time, but it just didn't showed up after renaming. My PVD is heavily customized, and I use my own skin (please check the screenshot enclosed), so I'm pretty aware of all aspects of renaming/updating fields. I tried also with IMDb User Reviews/Comments (my custom field name is different), but again to no avail. Simply said - total mystery to me.
5. Fields that I wanted to update/overwrite were: Description, IMDb User Reviews/Comments, Title (not origtitle), Top 250, IMDb Votes, Box Office, Actors, Director.

Kind regards and thank you for your time!
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 02, 2018, 08:15:05 am
Quote
imdb.com/title/tt0034583/
 imdb.com/title/tt0082176/
 imdb.com/title/tt0070518/
 imdb.com/title/tt0074252/

Casablanca (1942)
https://www.imdb.com/title/tt0034583/

Christiane F. - Wir Kinder vom Bahnhof Zoo (1981)
https://www.imdb.com/title/tt0082176/

Pat Garrett & Billy the Kid (1973)
https://www.imdb.com/title/tt0070518/

Brutti, sporchi e cattivi (1976)
https://www.imdb.com/title/tt0074252/
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 02, 2018, 08:16:44 am
Quote
I mean, and I'm not sure at all, when we started to use IMDb scripts I noticed that I have duplicate entries for people. It's like some people were imported by nostra's imdb.dll and their duplicates with scripts. So now, for example I have two (sometimes, even more) actors with the same name in database and their filmography is split (to movies imported by imdb.dll and other part with script).

There is a possibility that duplicates are people for certain reasons, such as different url addresses (or they are not at all) for people. I have already noticed this in my IMDB records for people. For  the IMDB plugin was the download URL of the Imdb reference pages without " www " in the url address. IMDB script has downloaded a full url with Main (overview) Imdb pages.

Finally, I had a similar problem with the TheMovieDB [API] script, where there was a problem with url titles for the Director because the script code was incorrectly written in the script and there were duplicate names for the director because of this, but not for all, only in certain cases.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on October 03, 2018, 12:13:40 pm
I have found different url formats versions in my PVD too?
Can we write a fast url standards?
My proposal is (remember that we cann't use httpS):
IMDB Movie (finish with '/'=Slash):
TMDB (NO finish with '/'=Slash):
FilmAffinity (finish with '.html', maybe spanish (several countries) or english):
BTW, in the last times of imdb.dll pluging it retrived bad movie and people url (for instance for directors with URL+'?ref_=tt_ov_dr' and writers URL+'?ref_=tt_ov_wr'). In that moment I wrote a little SQL script (you can use it in debug PVD mode in one hidden option of Help menu) in order to correct the database:
                 update PEOPLE set "url"=replace("url",'?ref_=tt_ov_dr', '/');
                 update PEOPLE set "url"=replace("url",'?ref_=tt_ov_wr', '/');
perhaps it may help to correct something.

I just wanted to ask if it's possible FA to have it's own rating custom field?
In the next MOD version I will include the solution that Ivek was advancing
 AddCustomFieldValueByName('FilmAffinity_Rating',ItemValue);
because it doesn't nothing bad for the "normal" users but you need add by yourself this Custom field.

BTW, I have included several 'hidden' Custom Fields of Ivek23 in the scripts.
Ivek23, perhaps can be useful for other users open a new Thread with the information "Possibles improving Custom Fields" working in MOD version, how is the information and how add in the PVD database.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: afrocuban on October 03, 2018, 09:06:50 pm
Thank you for your feedback, guys. Good to know that I'm not the one who messed up database, since you two have also people duplicates. Thanks for FA rating and all "hidden" custom fields.
I tried your SQL script in debug mode, cause I also notice mistakes in imported urls. Thank you for that. It didn't want to run with "As script" unchecked (some dynamic error...) so I ran it "As script", and although my database is which is over 15GB, with posters it finished in 3-4 minutes claiming "Script successfully executed"
After that I tried to update movies I stated above, but no luck. But, for example, I just repeated update for "All About Eve" which didn't succeed at first, and now it succeeded?! It never happened in original PVD, and the only conclusion I can make is that if some field isn't rewritable (for God knows what reason) the record cannot be updated.

I'd appreciate any further help on this and am more than grateful for your tremendous work and efforts to make PVD alive!

My best regards to you two.

Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on October 04, 2018, 10:32:43 am
Have you used the Tools/Optimize database ... ?
Perhaps it solved your "write protected" records but I warm you that it spend a lot, a lot of time.
With a 7.000 movies you must have more of 200.000 persons!
Perhaps you will spent one or two days with PVD working and the PC un-responding.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: afrocuban on October 04, 2018, 04:00:07 pm
Thanks for your idea, VVV. I remember I used that option at the very beginning, and that then as I can remember it deleted some records I didn't want to. Do you by any chance know what exactly optimizing will do to my database? I hae never ever had some problem with PVD database speed, but whatever - I'll do it if my records will not be deleted or database corrupted.

Kind regards
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: afrocuban on October 06, 2018, 12:00:34 pm
Dear VVV and Ivek,

Meanwhile, I have experienced another strange issue. While updating some movies, it asks me to confirm the movie, although IMDb urls are already in database, and for for twice!. You can check comparing offered url on the first screenshot (question marks), with the url already in database in the second screenshot (red arrow) - they are the same.

Also, for some movies I get infobox "Movie not found :)" although IMDb url exists - please see third screenshot?!

It didn't happen earlier.

It's like script doesn't use existing urls. Why is that?

Any idea? Thank you in advance.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on October 06, 2018, 01:38:02 pm
This is easy: in both "13" and "After life"  imdb url's lacks "www" remeber the standar url (http://www.videodb.info/forum_en/index.php/topic,4134.msg20759.html#msg20759) is:
http://www.imdb.com/title/ttxxxxxxx/

Perhaps you can use the SQL script:

update MOVIES set "url"=replace("url",'http://imdb', 'http://www.imdb');
.

BTW, one tools that can help you is pass the db to Excel with the export template "CSVDB" and search with the excel filters some bizarre things.
Quote
Do you by any chance know what exactly optimizing will do to my database?
It will do as you select: delete duplicates entries, delete orphan persons (persons not related with your movies). You can select if it will do with all movies/people or only qith the visible (so you can protect movies with a filter and hidde it's)
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: afrocuban on October 06, 2018, 03:08:55 pm
Thanks VVV. It's clear now.

Is it possible then, considering your statement:

I have found different url formats versions in my PVD too?
Can we write a fast url standards?
My proposal is (remember that we cann't use httpS):
IMDB Movie (finish with '/'=Slash):
  • Movie=http://www.imdb.com/title/ttxxxxxxx/
  • Person=http://www.imdb.com/name/nmxxxxxxx/
TMDB (NO finish with '/'=Slash):
  • Movie=http://www.themoviedb.org/movie/xxxxxxx
  • Person=http://www.themoviedb.org/person/xxxxxxx
FilmAffinity (finish with '.html', maybe spanish (several countries) or english):
  • Movie Spanish= http://www.filmaffinity.com/es/filmxxxxxxx.html
  • Movie English= http://www.filmaffinity.com/en/filmxxxxxxx.html


... to standardize urls in form:

[*.]imdb.com/title/ttxxxxxxx/
[*.]imdb.com/name/nmxxxxxxx/
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 06, 2018, 06:01:44 pm
Dear VVV and Ivek,

Meanwhile, I have experienced another strange issue. While updating some movies, it asks me to confirm the movie, although IMDb urls are already in database, and for for twice!. You can check comparing offered url on the first screenshot (question marks), with the url already in database in the second screenshot (red arrow) - they are the same.

Also, for some movies I get infobox "Movie not found :)" although IMDb url exists - please see third screenshot?!

It didn't happen earlier.

It's like script doesn't use existing urls. Why is that?

Any idea? Thank you in advance.

I myself have written here (http://www.videodb.info/forum_en/index.php/topic,4134.msg20758.html#msg20758) why this happens, and who is responsible for problems with IMDB url addresses. This is the IMDB plugin, which caused problems to the afrocuban and also to me, but I edited it manually by adding " www " letters in the IMDB url.

There is a possibility that duplicates are people for certain reasons, such as different url addresses (or they are not at all) for people. I have already noticed this in my IMDB records for people. For  the IMDB plugin was the download URL of the Imdb reference pages without " www " in the url address. IMDB script has downloaded a full url with Main (overview) Imdb pages.

Finally, I had a similar problem with the TheMovieDB [API] script, where there was a problem with url titles for the Director because the script code was incorrectly written in the script and there were duplicate names for the director because of this, but not for all, only in certain cases.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on October 06, 2018, 08:39:06 pm
Quote
Is it possible then, considering your statement:
... to standardize urls in form:
[*.]imdb.com/title/ttxxxxxxx/
[*.]imdb.com/name/nmxxxxxxx/

I think to do that this is dificult because the script use the parameter
BASE_URL_PRE       = 'http://www.imdb.com/title/''
to make the url and for search if exist the url.

And this is risky because if the original PVD came to live again it use the same behavior with the Function GetBaseURL for locate the url.

But Ivek23 and you don't need add 'www' manual the SQL script (remeber, in debug mode Help/SQL, As script checked) 
update MOVIES set "url"=replace("url",'http://imdb', 'http://www.imdb');

works fine: I test it.

Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: afrocuban on October 06, 2018, 11:46:22 pm
Hello VVV,

Thank you for another SQL tip. I applied it immediately upon you first right about it, and it worked like a charm! I just thought if it's possible to generalize the term as much as possible, because we don't know what will happen with IMDb code in the future and with maintain of PVD and IMDb script, but hey - we will survive. I have my database for 20 years, from the first days of "WhereIsIt?" program, til final solution - PVD.

About optimizing database, thank you for explanation, but I think I mustn't use it because it looks it will do exactly what I don't want at any cost - it'll delete duplicate people, so I would loose them in movies, so I would again have to update 7000 movies to get people back, again most probably to get duplicates. The only solution to duplicates that I see is to delete all people, then to update all movies with people records (director, actor, producer, writer, composer, etc...).

Maybe I'd do that, but only when I'm sure that I will be able to update movies without breaks that I have now.

To add to this, I hve experienced one more IMDb script issue:
- If URL doesn't have slash - "/" at the and, we get "404" error message, like URL doesn't exist on imdb.com, while actually url is valid. Please see first screenshot attached.
- After pressing OK (1-5 times, depending on how many pages are imported), script adds ghost link: - "http:/"? Please check second screenshot.

Anything we can do about this?

Best regards
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 07, 2018, 08:31:04 am
Now I've tried to update The Villain movie title, where there is already information and has such an url address.

http://imdb.com/title/tt0080097/

The interesting thing is that I did not have any problems in updating the information.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 07, 2018, 09:53:59 am
BTW, I have included several 'hidden' Custom Fields of Ivek23 in the scripts.
Ivek23, perhaps can be useful for other users open a new Thread with the information "Possibles improving Custom Fields" working in MOD version, how is the information and how add in the PVD database.

To that, when I do some more tests, because I discovered some more errors and I still test some more improvements to the code sections for IMDB_ [EN] [HTTPS] script.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on October 07, 2018, 10:44:54 am
Quote from: Ivek23
The interesting thing is that I did not have any problems in updating the information.
Is sure that the script don't find the IMDB url. Maybe that the script are always searching the movie URL's in IMDB page but it find at first and don't ask nothing. You can see that with the Log tool. In this movies, after pass the script, don't you have two IMDB url's stored?

Quote from: afrocuban
To add to this, I hve experienced one more IMDb script issue:
- If URL doesn't have slash - "/" at the and, we get "404" error message, like URL doesn't exist on imdb.com, while actually url is valid. Please see first screenshot attached.
- After pressing OK (1-5 times, depending on how many pages are imported), script adds ghost link: - "http:/"? Please check second screenshot.
This is not easy to solve with a SQL script so I code a not critical MOD patch for you. In the next post.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on October 07, 2018, 10:47:29 am
Attached non critical Acumulative Scripts Patch date:2018.10.07 (to copy in PVD folder for overwriting the existing script files).

Changelog:
IMDB_[EN][HTTPS].psf and IMDB_People_[EN][HTTPS].psf:

 V 1.2.0.2 (07/10/2018) Managed IMDB url's not finish in BASE-URL-SUF='/' and principal page download errors.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 07, 2018, 11:30:25 am
Quote from: Ivek23
The interesting thing is that I did not have any problems in updating the information.
Is sure that the script don't find the IMDB url. Maybe that the script are always searching the movie URL's in IMDB page but it find at first and don't ask nothing. You can see that with the Log tool. In this movies, after pass the script, don't you have two IMDB url's stored?

I'm sure and I stand for it, because the next url did not cause any problems when I updated the information. I checked this several times already due to the afrocuban major problems described in this regard.

There Was A Crooked Man...       http://imdb.com/title/tt0066448/

No, I do not have any duplicate URLs.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on October 07, 2018, 12:23:30 pm
I'm agree with you that the script doesn't give you any problems.
I want to say that perhaps this is because the script don't use the stored URL and it search the title in IMDB and continues scrapping.
You may see the script behaviour with the log window tool.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: afrocuban on October 07, 2018, 08:25:41 pm
Thank you so much for your help guys! I will try cumulative script patch after finish updating.
Regardless of Ivek's and mine issue regarding "www", it is obviously solvable, one way or another. Still, the original, first issue is mystery and I connected it with the issue that I cannot overwrite some fields (mostly "Title", but some other as well), manually or automatically. Do you have any idea why some field could not be editied, manually, or automatically???

Thanks in advance!
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: afrocuban on October 08, 2018, 01:18:43 am
I have found the cause of the error from this message:
http://www.videodb.info/forum_en/index.php/topic,4134.msg20748.html#msg20748

... and the workaround for it.

The problem is IMDb url that is stored in the "URL" field! Although it is perfectly valid (I copy it from the URL field then paste it in browser and it goes to the proper movie), it will not work if you try to update movie in PVD, or to manually edit any other field imported by IMDb script or plugin!

So, I went to Grid View, and for some movies I tried to delete Title and Origititle and, bam - I got the same error message as the one while importing with script.

Finally, in Grid view I deleted stored IDMb url (which is perfectly valid), and then let the script imports data and, another BAM - everything worked great!

Meaning, when I delete stored IMDb url (it is sometimes possible in Tree View, but it is always possible in Grid View), I am able to edit any other field, hence to update movie!

Why is this happening and why to only certain movies/records it's the mystery probably no one will solve.

If someone can explain this I'd be more than grateful. Luckily, I had only 105 movies of 7000 movies with this issue, so I will manually edit each one url and update it, but that will not resolve possible future issues. Workaround is not a solution, that is why it is called only - workaround.

Best regards
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on October 08, 2018, 07:41:16 am
Good news even if it must expend "some" manual editing time! It's true that the Grid View edition is a little bit forgotten.
Perhaps you have some especial chars in the URL's fields that the browser is capable to manage and PVD crash (for instance, the URL's must be separated by spaces not by Return, char#13).

Quote
Regardless of Ivek's and mine issue regarding "www", it is obviously solvable, one way or another.
I strongly recommend you use the standar url (http://www.videodb.info/forum_en/index.php/topic,4134.msg20759.html#msg20759) formats because there is other script (TheMovieDB) that use the IMDB link and is the normal way of PVD to detect the provider url (remember when PVD directly download the page when it detects the BASE_URL in the URL field, this is the reason that in the HTTPS scripts the function GetBaseURL returns BASE_URL_ROUNDABOUT in order to cheat GET PVdB funtion and rondabout the "https" fail).

And if all records are with those formats are easy to change with a SQL script because the URL field is a little bit especial, for instance, I haven't found the manner to protect it from script write or to delete all with the mass editor.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 10, 2018, 08:05:36 am
IMDB_ [EN] [HTTPS] script

A beauty correction for a piece of code for Date ~ Updated ~ (choose simple or verbose version) .

Quote
    //Date ~Updated~ (choose simple or verbose version)
        Date:=DateToStr(CurrentDateTime);
        //AddFieldValueXML('viewdate',Date); //Only date, don't admit time-. Set Seen value at the same time.
        //AddFieldValueXML('moddate',Date+' '+TimeToStr(CurrentDateTime)); //Block the dB saving   
        ExplodeString(Date,DateParts,'-');
        Date:=DateParts[2]+'.'+ DateParts[1]+'.'+DateParts[0]; 
        Date:=StringReplace(Date,'01.','1.',True,True,False);
      Date:=StringReplace(Date,'02.','2.',True,True,False);
      Date:=StringReplace(Date,'03.','3.',True,True,False);
      Date:=StringReplace(Date,'04.','4.',True,True,False);
      Date:=StringReplace(Date,'05.','5.',True,True,False);
      Date:=StringReplace(Date,'06.','6.',True,True,False);
      Date:=StringReplace(Date,'07.','7.',True,True,False);
      Date:=StringReplace(Date,'08.','8.',True,True,False);
      Date:=StringReplace(Date,'09.','9.',True,True,False);   
      
        //AddCustomFieldValueByName('Updated',Date); // Simple
        //AddCustomFieldValueByName('Updated',Date+' at '+TimeToStr(CurrentDateTime)); // Verbose
        AddCustomFieldValueByName('Updated',Date+' at '+TimeToStr(CurrentDateTime)+' • '+SCRIPT_NAME+' '+SCRIPT_VERSION);  // Annoying


For Function ParsePage_IMDBMovieMPAA is also a small correction again.

Quote
Function ParsePage_IMDBMovieMPAA(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError;  If any big problem? with exit
    //Retrieve: ~MPAA~
  Var
    curPos:Integer;
    mpaaValue:String;
  Begin
.
.
.
    mpaaValue:=StringReplace(mpaaValue,'Suggested Rating','. Suggested Rating',False,False,False); //Avoid union in films with 'Suggested Rating'
   mpaaValue:=StringReplace(mpaaValue,'update: ','',False,False,True);                         //Cleanig value. //WEB_SPECIFIC
   mpaaValue:=StringReplace(mpaaValue,': ','',False,False,True);                         //Cleanig value. //WEB_SPECIFIC
    curPos:=Pos('Edit',mpaaValue);                                     //WEB_SPECIFIC.
    If 0<curPos then mpaaValue:=Copy(mpaaValue,0,curPos-1);   
    If (mpaaValue='Approved') Then mpaaValue:='Rated G';
.
.
.
    AddFieldValueXML('mpaa',mpaaValue);
    LogMessage('      Get result MPAA='+mpaaValue+'||');
    LogMessage('Function ParsePage_IMDBMovieMPAA END=====================||');
  End; //BlockClose


Full Cast and Crew section

For directors is also a small correction again.


Quote
    //Go to "Directed by" ~crew~ctDirectors
    curPos:=Pos('<h4 class="dataHeaderWithBorder">Directed by',HTML);                     //Strings start which opens the block content data. WEB_SPECIFIC   
    If 0<curPos Then Begin
      curPos:=PosFrom('</h4>',HTML,curPos);                                              //Strings end which opens the block content data.  WEB_SPECIFIC
      curPos:=curPos+Length('</h4>');                                                    //Strings end which opens the block content data.  WEB_SPECIFIC
      //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~ n the ~crew~
      endPos:=PosFrom('</table>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
      ItemList:=Copy(HTML,curPos,endPos-curPos);
      //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');
      curPos:=Pos('<td class="name">',ItemList)                                         //String which opens the subList data. WEB_SPECIFIC
      index:=1;
      While curPos>0 Do Begin
         If (index>PEOPLE_LIMIT) Then break;     //Limited depassed (Remember index begin in 0).
         //Get PersonURL (Always must it has)
         PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF;     //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
         //Get Name (Always must it has)
         Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC     
         LogMessage('      Parse Results Name:'+Name+'||');
         //Get Role: PVD don't save Role in crew people but is added for Ivek23 as LogMessage compatibility
         //Role:=TextBetWeen(ItemList,'<td class="credit">','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC
            Role:=TextBetWeen(ItemList,'<td ','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC         
         LogMessage('      Parse Results Role:'+Role+'||');
         AddMoviePerson(Name,'','',PersonURL,ctDirectors);
         LogMessage('      Get results Director:#'+IntToStr(index)+'|'+Name+'|'+PersonURL+'||ctDirectors');
         curPos:=PosFrom('<td class="name">',ItemList,curPos)                                            //String which opens the subList data. WEB_SPECIFIC
         index:=index+1;
      End;
    End;
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 10, 2018, 09:30:05 am
IMDB_ [EN] [HTTPS] script

Function ParsePage_IMDBMovieBASE

Some corrections and new code sections for Function ParsePage_IMDBMovieBASE

Get ~studio~ "Production Co" (several values in a comma separated list)


Quote
    //Get ~studio~ "Production Co" (several values in a comma separated list)
    curPos:=Pos('<h4 class="inline">Production Co:</h4>',HTML);                                      //WEB_SPECIFIC.IC.
    If 0<curPos Then Begin       
       ItemValue:=TextBetWeen(HTML,'<h4 class="inline">Production Co:</h4>','</span>',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=StringReplace(ItemValue,'See more »','',True,False,True);      //Cleanning values
       ItemValue:=StringReplace(ItemValue,'See more','',True,True,False);
       //ItemValue:=StringReplace(ItemValue,', The','',True,False,True);
       //ItemValue:=StringReplace(ItemValue,'The, ','',True,False,True);
       //ItemValue:=StringReplace(ItemValue,'The ','',True,False,True);   
   
       AddFieldValueXML('studio',ItemValue);
       LogMessage('      Get results Studio/Production Co:'+ItemValue+'||');
    End;


Get ~mpaa~. GET_FULL_MPAA = False only the info of the principal movie page.

Quote
    //Get ~mpaa~. GET_FULL_MPAA = False only the info of the principal movie page.
    //If Not(GET_FULL_MPAA) Then Begin
       //The text can be "Certificate:" or "Motion Picture Rating" but alway after genres.
    //   If 0<curPos Then Begin
    //      ItemValue:=TextBetWeen(HTML,'<span itemprop="contentRating">','</span>',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
    //      AddFieldValueXML('mpaa',ItemValue);
    //      LogMessage('      Get result mpaa:'+ItemValue+'||');
    //   End;
    //End;
    //Get ~mpaa~. GET_FULL_MPAA = False only the info of the principal movie page.
    If Not(GET_FULL_MPAA) Then Begin
       //The text can be "Certificate:" or "Motion Picture Rating" but alway after genres.
      curPos:=Pos('<h4 class="inline">Certificate:</h4>',HTML);
       If 0<curPos Then Begin
          ItemValue:=TextBetWeen(HTML,'<h4 class="inline">Certificate:</h4>','<span class="see-more inline">',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC             
        //ItemValue:=StringReplace(ItemValue,'See all certifications »','',True,False,True);
        ItemValue:=StringReplace(ItemValue,'    |','',True,False,True);       
          AddFieldValueXML('mpaa',ItemValue);     //AddFieldValue(mfMPAA,ItemValue);
          If ItemValue <> '' then LogMessage('      Get results MPAA Certificate: '+ItemValue+' ||');
       End;
    End;   
    //Get ~mpaa~. GET_FULL_MPAA = False only the info of the principal movie page.
    If Not(GET_FULL_MPAA) Then Begin
       //The text can be "Certificate:" or "Motion Picture Rating" but alway after genres.
      curPos:=Pos('<h4>Motion Picture Rating',HTML);
      If 0<curPos Then Begin
          ItemValue:=TextBetWeen(HTML,'<span>','</span>',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC       
          AddFieldValueXML('mpaa',ItemValue);
          LogMessage('      Get results MPAA: '+ItemValue+' ||');
       End;
    End;   

The first part of the Get ~ mpaa ~ code. (GET_FULL_MPAA = False only the info on the main movie page.) does not work, so I added working parts of the code.

Get ~script info~

Quote
Function ParsePage_IMDBMovieBASE(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit;
    //Retrieve: ~title~, ~year~, ~origtitle~, ~poster~ / ~imdbrating~, ~IMDB_Votes~ (Custom Field) / ~TOP_250~(Custom Field) /
    //          If Not(GET_FULL_CREDIT): ~crew~ctDirectors,ctWriters,ctComposers,ctProducers(Not in base page), ctActors
    //         ~description~ / ~category~ "keywords" / ~tagline~ / ~genre~
    //         If Not(GET_FULL_MPAA) ~mpaa~
    //         ~country~ / ~rdate~ in contry provider local IP geolocation
    //         If Not(GET_FULL_AKA) ~aka~.
    //         ~budget~ / ~money~ / ~studio~ "Production Co"
    //         If Not(GET_FULL_FEATURES) ~features~
  Var
      curPos,endPos,index:Integer;
      ItemValue,ItemList,ImageFile:String;
     ItemValue1,ItemList1:String;
      titleValue:String;
      Name,Role,PersonURL:String;
     ReleaseDate:String;
     ReleaseDateParts: TWideArray;
    
  Begin
.
.
.
    //Get ~script info~
    curPos:=PosFrom('<script type="application/ld+json">{',HTML,curPos);
    //curPos:=curPos+Length('<script type="application/ld+json">{');
    endPos:=PosFrom('}</script>',HTML,curPos)+10;
    ItemList1:=Copy(HTML,curPos,endPos-curPos);
    //ItemList1:=RemoveTags(ItemList1, False);
    //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList1 script info:'+ItemList1+'||');
    ItemList:=TextBetWeenFirst(HTML,'<script type="application/ld+json"','}</script>'); //WEB_SPECIFIC.
    If (Length(ItemList)>0) Then Begin                             
      ItemValue:=TextBetWeenFirst(ItemList,'"@type": "','",');   //Strings which opens/closes the data. WEB_SPECIFIC
      ItemValue:=StringReplace(ItemValue,'TVSeries','TV Series',True,False,True);
      //AddCustomFieldValueByName('IMDB_Movietype',ItemValue);                                 
      if ItemValue <> '' then LogMessage('      Get result @type: '+ItemValue+' ||');
      ItemValue:=TextBetWeenFirst(ItemList,'"contentRating": "','",');   //Strings which opens/closes the data. WEB_SPECIFIC
      //AddCustomFieldValueByName('IMDB_MPAA',ItemValue);                                 
      if ItemValue <> '' then LogMessage('      Get result contentRating: '+ItemValue+' ||');
      ReleaseDate:=TextBetWeenFirst(ItemList,'"datePublished": "','",');   //Strings which opens/closes the data. WEB_SPECIFIC
      //if ReleaseDate <> '' then LogMessage('      Get result Release_Date_Published: '+ReleaseDate+' ||');
        ReleaseDate:=StringReplace(ReleaseDate,'-01','-1',True,False,True);
        ReleaseDate:=StringReplace(ReleaseDate,'-02','-2',True,False,True);
        ReleaseDate:=StringReplace(ReleaseDate,'-03','-3',True,False,True);
        ReleaseDate:=StringReplace(ReleaseDate,'-04','-4',True,False,True);
        ReleaseDate:=StringReplace(ReleaseDate,'-05','-5',True,False,True);
        ReleaseDate:=StringReplace(ReleaseDate,'-06','-6',True,False,True);
        ReleaseDate:=StringReplace(ReleaseDate,'-07','-7',True,False,True);
        ReleaseDate:=StringReplace(ReleaseDate,'-08','-8',True,False,True);
        ReleaseDate:=StringReplace(ReleaseDate,'-09','-9',True,False,True);
      if ReleaseDate <> '' then LogMessage('      Get result ReleaseDatePublished: '+ReleaseDate+' ||');      
      if ReleaseDate <> '' then begin
            ExplodeString(ReleaseDate,ReleaseDateParts,'-');
            ReleaseDate:=ReleaseDateParts[2]+'.'+ ReleaseDateParts[1]+'.'+ReleaseDateParts[0];
         AddCustomFieldValueByName('IMDB Release Date',ReleaseDate);   
         AddFieldValueXML('rdate',ReleaseDate);         
      if ReleaseDate <> '' then LogMessage('      Get result datePublished: '+ReleaseDate+' ||');      
      End;
      ItemValue:=TextBetWeenFirst(ItemList,'"ratingCount": ',',');   //Strings which opens/closes the data. WEB_SPECIFIC
      //AddCustomFieldValueByName('IMDB Votes',ItemValue); 
      //AddCustomFieldValueByName('IMDB Votes:',ItemValue);
      AddCustomFieldValueByName('IMDB_Votes',ItemValue);      
      if ItemValue <> '' then LogMessage('      Get result ratingCount: '+ItemValue+' ||');   
      ItemValue:=TextBetWeenFirst(ItemList,'"ratingValue": "','"');   //Strings which opens/closes the data. WEB_SPECIFIC
      AddFieldValueXML('imdbrating',ItemValue);
      AddCustomFieldValueByName('IMDB Rating',ItemValue);
      //AddCustomFieldValueByName('IMDBRating',ItemValue);
      if ItemValue <> '' then LogMessage('      Get result ratingValue: '+ItemValue+' ||');         
    End;   

I added a Get ~ script info ~ code, where imdbrating and IMDB_Votes, which should also work if there is a change to the code for Get ~ imdbrating ~, ~ IMDB_Votes ~ .

I also added the ReleaseDate code, which can add a missing release date or a more correct release date. In the settings you can adjust to add this information to you only when it is missing or overwritten.  I did this because several times this was not the correct original release date,
which I would like to have in my database. Many times I get the release date record when this release date was published in my country.

IMDB_ [EN] [HTTPS] _000 script is attached.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: afrocuban on October 10, 2018, 09:40:09 pm
Dear VVV,

Some of my URL fields probably had some special characters in IMDb addreses, because now after I manually deleted url, than import data from imdb, I got new imdb url in url field. To test this newly imported imdb url, I applied update again with "overwrite" option on and everything went well, so obviously you were right.

Thanks Ivek for the update.

I have discovered one more issue with both IMDB_[EN][HTTPS].psf and IMDB_[EN][HTTPS] _000.psf

I can update Series record (for example http://www.imdb.com/title/tt0863046/). But I can't update it's episodes. Updating just crashes PVD.

Here's part of the debug code

Code: [Select]
allocated memory  : 99,05 MB
command line      : viddb.exe -portable -debug
executable        : viddb.exe
exec. date/time   : 2018-08-08 10:01
version           : 1.0.2.7
compiled with     : Delphi 2010
madExcept version : 3.0l
callstack crc     : $0811da24, $53dbdf84, $4ecf9cfa
exception number  : 7
exception class   : Unknown
exception message : Unknown.

main thread ($9a4):
0811da24 +000 ???
008d0d77 +a27 viddb.exe    MainU    8520 +148 TPVDMain.ExecWebImport
008c5470 +3c8 viddb.exe    MainU    5445  +60 TPVDMain.DoPluginExecute
008cc7cf +057 viddb.exe    MainU    7482  +10 TPVDMain.ExecImpBtnClick
00551163 +06f viddb.exe    Controls           TControl.Click
005dc454 +000 viddb.exe    Buttons            TSpeedButton.Click
005dc43e +0ea viddb.exe    Buttons            TSpeedButton.MouseUp
00551598 +038 viddb.exe    Controls           TControl.DoMouseUp
00551614 +070 viddb.exe    Controls           TControl.WMLButtonUp
0055151e +07e viddb.exe    Controls           TControl.WMMouseMove
00550bf8 +2d4 viddb.exe    Controls           TControl.WndProc
0055081c +024 viddb.exe    Controls           TControl.Perform
00554de8 +0ac viddb.exe    Controls           TWinControl.IsControlMouseMsg
00555338 +3e4 viddb.exe    Controls           TWinControl.WndProc
00554b5c +02c viddb.exe    Controls           TWinControl.MainWndProc
004a9b5c +014 viddb.exe    Classes            StdWndProc
755f7885 +00a USER32.dll                      DispatchMessageW
005812c9 +11d viddb.exe    Forms              TApplication.ProcessMessage
0058130e +00a viddb.exe    Forms              TApplication.HandleMessage
00581639 +0c9 viddb.exe    Forms              TApplication.Run
009af241 +b69 viddb.exe    viddb     257 +120 initialization
75e03368 +010 kernel32.dll                    BaseThreadInitThunk

thread $1828 (TWorkerThread):
77ccf8da +0e ntdll.dll                           NtWaitForSingleObject
775015c8 +92 KERNELBASE.dll                      WaitForSingleObjectEx
75e0118f +3e kernel32.dll                        WaitForSingleObjectEx
75e01143 +0d kernel32.dll                        WaitForSingleObject
005a2651 +19 viddb.exe      VirtualTrees 6002 +3 TWorkerThread.Execute
00467507 +2b viddb.exe      madExcept            HookedTThreadExecute
004a703a +42 viddb.exe      Classes              ThreadProc
00406c38 +28 viddb.exe      System        985 +0 ThreadWrapper
004673e9 +0d viddb.exe      madExcept            CallThreadProcSafe
00467453 +37 viddb.exe      madExcept            ThreadExceptFrame
75e03368 +10 kernel32.dll                        BaseThreadInitThunk
>> created by main thread ($9a4) at:
005a2596 +16 viddb.exe      VirtualTrees 5965 +1 TWorkerThread.Create

thread $1e90:
77cd0166 +0e ntdll.dll     NtWaitForMultipleObjects
75e03368 +10 kernel32.dll  BaseThreadInitThunk

thread $1d80:
77ccf8da +0e ntdll.dll       NtWaitForSingleObject
775015c8 +92 KERNELBASE.dll  WaitForSingleObjectEx
75e0118f +3e kernel32.dll    WaitForSingleObjectEx
75e01143 +0d kernel32.dll    WaitForSingleObject
6a1a29b8 +38 MSVCR80.dll     _endthreadex
75e03368 +10 kernel32.dll    BaseThreadInitThunk

thread $1590: <priority:2>
77ccf8da +0e ntdll.dll       NtWaitForSingleObject
775015c8 +92 KERNELBASE.dll  WaitForSingleObjectEx
75e0118f +3e kernel32.dll    WaitForSingleObjectEx
75e01143 +0d kernel32.dll    WaitForSingleObject
6a1a29b8 +38 MSVCR80.dll     _endthreadex
75e03368 +10 kernel32.dll    BaseThreadInitThunk

thread $15ec:
77cd1f4f +0b ntdll.dll     NtWaitForWorkViaWorkerFactory
75e03368 +10 kernel32.dll  BaseThreadInitThunk

disassembling:
[...]
008d0d67        mov     eax, [$9cb0b8]
008d0d6c        mov     eax, [eax]
008d0d6e        mov     eax, [eax+$30]
008d0d71        call    -$4c988e ($4074e8)     ; System.@LStrToPChar
008d0d76        push    eax
008d0d77      > call    dword ptr [ebp-$18]
008d0d7a        movsx   esi, ax
008d0d7d 8521   sub     esi, 1
008d0d80        jb      loc_8d17aa
008d0d86        jz      loc_8d0d9d
008d0d88        dec     esi

I suspect it has something to do with double quotes  - ". When " is in title (for the episode, importing is tried from the page which title is series name under the double qoutes, then episode name, then PVD crashes.

For example, for the http://www.imdb.com/title/tt1006987/  ,
script tries to import data from imdb page which title is (can be seen on browser tab) "Flight of the Conchords": Sally, or something similar, but it definitely has double quotes,

Hopefully I can get some advise.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 13, 2018, 06:37:07 am
BTW, I have included several 'hidden' Custom Fields of Ivek23 in the scripts.
Ivek23, perhaps can be useful for other users open a new Thread with the information "Possibles improving Custom Fields" working in MOD version, how is the information and how add in the PVD database.

To that, when I do some more tests, because I discovered some more errors and I still test some more improvements to the code sections for IMDB_ [EN] [HTTPS] script.

You can also find more information about Custom Fields in the topic Possibles improving Custom Fields working in MOD version. (http://www.videodb.info/forum_en/index.php/topic,4141.msg20789.html#msg20789)
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 13, 2018, 07:38:12 am
In the event that other interesting movie titles appear in the search for a particular movie, and you also want to have this information, the URL field will be downloaded for the requested address as http://www.imdb.com/title/ttxxxxxxx/ url address for the first marked movie title.
For the following tagged titles titles titles are recorded in the url field such: http://httpbin.org/response-headers?key=http://www.imd/title/ttxxxxxxx/ url title for other movie titles.

That's why I am now using this SQL script
Perhaps you can use the SQL script:

update MOVIES set "url"=replace("url",'http://imdb', 'http://www.imdb');
has been repaired and is now:

update MOVIES set "url"=replace(("url",'http://httpbin.org/response-headers?key=http://www.imdb', 'http://www.imdb');

Now, such a SQL script successfully addresses these URLs and are now corrected to such a format:
http://www.imdb.com/title/ttxxxxxxx/


I hope that users will fix certain URLs if they have anywhere in the url field.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 13, 2018, 08:57:16 am
Thanks Ivek for the update.

Thanks.

Dear VVV,

Some of my URL fields probably had some special characters in IMDb addreses, because now after I manually deleted url, than import data from imdb, I got new imdb url in url field. To test this newly imported imdb url, I applied update again with "overwrite" option on and everything went well, so obviously you were right.

Thanks Ivek for the update.

I have discovered one more issue with both IMDB_[EN][HTTPS].psf and IMDB_[EN][HTTPS] _000.psf

I can update Series record (for example http://www.imdb.com/title/tt0863046/). But I can't update it's episodes. Updating just crashes PVD.

Here's part of the debug code

Code: [Select]
allocated memory  : 99,05 MB
command line      : viddb.exe -portable -debug
executable        : viddb.exe
exec. date/time   : 2018-08-08 10:01
version           : 1.0.2.7
compiled with     : Delphi 2010
madExcept version : 3.0l
callstack crc     : $0811da24, $53dbdf84, $4ecf9cfa
exception number  : 7
exception class   : Unknown
exception message : Unknown.

main thread ($9a4):
0811da24 +000 ???
008d0d77 +a27 viddb.exe    MainU    8520 +148 TPVDMain.ExecWebImport
008c5470 +3c8 viddb.exe    MainU    5445  +60 TPVDMain.DoPluginExecute
008cc7cf +057 viddb.exe    MainU    7482  +10 TPVDMain.ExecImpBtnClick
00551163 +06f viddb.exe    Controls           TControl.Click
005dc454 +000 viddb.exe    Buttons            TSpeedButton.Click
005dc43e +0ea viddb.exe    Buttons            TSpeedButton.MouseUp
00551598 +038 viddb.exe    Controls           TControl.DoMouseUp
00551614 +070 viddb.exe    Controls           TControl.WMLButtonUp
0055151e +07e viddb.exe    Controls           TControl.WMMouseMove
00550bf8 +2d4 viddb.exe    Controls           TControl.WndProc
0055081c +024 viddb.exe    Controls           TControl.Perform
00554de8 +0ac viddb.exe    Controls           TWinControl.IsControlMouseMsg
00555338 +3e4 viddb.exe    Controls           TWinControl.WndProc
00554b5c +02c viddb.exe    Controls           TWinControl.MainWndProc
004a9b5c +014 viddb.exe    Classes            StdWndProc
755f7885 +00a USER32.dll                      DispatchMessageW
005812c9 +11d viddb.exe    Forms              TApplication.ProcessMessage
0058130e +00a viddb.exe    Forms              TApplication.HandleMessage
00581639 +0c9 viddb.exe    Forms              TApplication.Run
009af241 +b69 viddb.exe    viddb     257 +120 initialization
75e03368 +010 kernel32.dll                    BaseThreadInitThunk

thread $1828 (TWorkerThread):
77ccf8da +0e ntdll.dll                           NtWaitForSingleObject
775015c8 +92 KERNELBASE.dll                      WaitForSingleObjectEx
75e0118f +3e kernel32.dll                        WaitForSingleObjectEx
75e01143 +0d kernel32.dll                        WaitForSingleObject
005a2651 +19 viddb.exe      VirtualTrees 6002 +3 TWorkerThread.Execute
00467507 +2b viddb.exe      madExcept            HookedTThreadExecute
004a703a +42 viddb.exe      Classes              ThreadProc
00406c38 +28 viddb.exe      System        985 +0 ThreadWrapper
004673e9 +0d viddb.exe      madExcept            CallThreadProcSafe
00467453 +37 viddb.exe      madExcept            ThreadExceptFrame
75e03368 +10 kernel32.dll                        BaseThreadInitThunk
>> created by main thread ($9a4) at:
005a2596 +16 viddb.exe      VirtualTrees 5965 +1 TWorkerThread.Create

thread $1e90:
77cd0166 +0e ntdll.dll     NtWaitForMultipleObjects
75e03368 +10 kernel32.dll  BaseThreadInitThunk

thread $1d80:
77ccf8da +0e ntdll.dll       NtWaitForSingleObject
775015c8 +92 KERNELBASE.dll  WaitForSingleObjectEx
75e0118f +3e kernel32.dll    WaitForSingleObjectEx
75e01143 +0d kernel32.dll    WaitForSingleObject
6a1a29b8 +38 MSVCR80.dll     _endthreadex
75e03368 +10 kernel32.dll    BaseThreadInitThunk

thread $1590: <priority:2>
77ccf8da +0e ntdll.dll       NtWaitForSingleObject
775015c8 +92 KERNELBASE.dll  WaitForSingleObjectEx
75e0118f +3e kernel32.dll    WaitForSingleObjectEx
75e01143 +0d kernel32.dll    WaitForSingleObject
6a1a29b8 +38 MSVCR80.dll     _endthreadex
75e03368 +10 kernel32.dll    BaseThreadInitThunk

thread $15ec:
77cd1f4f +0b ntdll.dll     NtWaitForWorkViaWorkerFactory
75e03368 +10 kernel32.dll  BaseThreadInitThunk

disassembling:
[...]
008d0d67        mov     eax, [$9cb0b8]
008d0d6c        mov     eax, [eax]
008d0d6e        mov     eax, [eax+$30]
008d0d71        call    -$4c988e ($4074e8)     ; System.@LStrToPChar
008d0d76        push    eax
008d0d77      > call    dword ptr [ebp-$18]
008d0d7a        movsx   esi, ax
008d0d7d 8521   sub     esi, 1
008d0d80        jb      loc_8d17aa
008d0d86        jz      loc_8d0d9d
008d0d88        dec     esi

I suspect it has something to do with double quotes  - ". When " is in title (for the episode, importing is tried from the page which title is series name under the double qoutes, then episode name, then PVD crashes.

For example, for the http://www.imdb.com/title/tt1006987/  ,
script tries to import data from imdb page which title is (can be seen on browser tab) "Flight of the Conchords": Sally, or something similar, but it definitely has double quotes,

Hopefully I can get some advise.

As far as I can quickly figure out what causes crashes of PVD. This is the complete code for Also Known As (AKA). I attach IMDB_ [EN] [HTTPS] (episodes) script, which should fix this problem because I blocked " GET_FULL_AKA = False; ".

IMDB_ [EN] [HTTPS] (episodes) script has been added.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: afrocuban on October 13, 2018, 10:04:41 am
Yes! It seems that so far works great, dear Ivek (I'm still updating episodes)!!! Thank you so much.

So, of all of my 7000 movies, it left only one movie-mystery that I cannot update: Mr. Magorium's Wonder Emporium (https://www.imdb.com/title/tt0457419/)

Whether IMDb url field is empty, or I manually put IMDb url in the field, all 3 scripts give the same error 404 - please check screenshot attached. I also tried and emptied all IMDb fields except Title, but no luck to import

Can anyone of you import data for this movie?
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 13, 2018, 06:43:12 pm
Yes! It seems that so far works great, dear Ivek (I'm still updating episodes)!!! Thank you so much.

Thanks.

So, of all of my 7000 movies, it left only one movie-mystery that I cannot update: Mr. Magorium's Wonder Emporium (https://www.imdb.com/title/tt0457419/)

Whether IMDb url field is empty, or I manually put IMDb url in the field, all 3 scripts give the same error 404 - please check screenshot attached. I also tried and emptied all IMDb fields except Title, but no luck to import

Can anyone of you import data for this movie?

I checked with IMDB_ [EN] [HTTPS] script, which was updated after August 20, it is not working, and the PVD or script reports an error saying that this url address does not exist. IMDB_ [EN] [HTTPS] script, which was in MOD V.2018.08.08, but limitedly transfers information. I found the error cause and this is located in the part of the script from Function GetDownloadURL to Function ParsePage_BingSearch. I replaced this code section in IMDB_ [EN] [HTTPS] script with a code section from IMDB_ [EN] [HTTPS] script, which was in MOD V.2018.08.08. This updated IMDB_ [EN] [HTTPS] script was easy and correctly downloaded all the information for this problematic movie title.

Updated IMDB_ [EN] [HTTPS] _000a script is attached.

VVV_Easy_Programing, please check out the difference between IMDB_ [EN] [HTTPS] _000 script and IMDB_ [EN] [HTTPS] _000a script in the part of the script code from Function GetDownloadURL to Function ParsePage_BingSearch.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: afrocuban on October 13, 2018, 08:45:43 pm
Thanks for the fast response dear Ivek. I have one question. Can your IMDB_[EN][HTTPS] _000a.psf script substitute all other IMDb scripts, except IMDB_[EN][HTTPS](episodes).psf of course?
Thank you for an answer in advance.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 14, 2018, 07:27:16 am
Thanks for the fast response dear Ivek. I have one question. Can your IMDB_[EN][HTTPS] _000a.psf script substitute all other IMDb scripts, except IMDB_[EN][HTTPS](episodes).psf of course?
Thank you for an answer in advance.

Yes, you can, as mentioned above. Also, keep IMDB_ [EN] [HTTPS] _000.psf script if there is any problem with IMDB_ [EN] [HTTPS] _000a.psf script, just in case, that you can also use it.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: afrocuban on October 14, 2018, 11:56:20 am
Thank you so much!
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 17, 2018, 08:15:45 am
Thank you so much!

Welcome.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 17, 2018, 08:22:26 am
IMDB_ [EN] [HTTPS] script

It is necessary to check " // Get Cast." GET_FULL_CREDIT = False only the info of the main movie page. " code in Function ParsePage_IMDBMovieBASE section, if it still works (as far as I know, does not work) and is needed in the script. If it does not work, this part of the code should be removed, except for actors, because it only burdens the work and the size of the script, but there is no transfer of information. It should be removed and replaced only with the actors code, which would be arranged so that the user would be able to download full information for actors from the Function ParsePage_IMDBMovieCREDIT section, or only information for actors from the Function ParsePage_IMDBMovieBASE section. This would be the same as for the IMDB plugin, where you can determine the transfer of information for actors from the main movie page or from the Full Cast & Crew site.

This is the corrected code for actors from the main movie page.
Quote
  //Retreive Data Config
  GET_MAINPAGE_ACTORS_CREDIT  = True ;  //Set to True to ensure MAINPAGE_ACTORS_CREDIT info is added. Otherwise: Set to False to ensure MAINPAGE_ACTORS_CREDIT info is not added. MAINPAGE_ACTORS_CREDIT info is in to the principal movie page.
  GET_FULL_ACTORS_CREDIT  = True ;  //Set to True to ensure FULL_ACTORS_CREDIT info is for Full Cast & Crew provider page. Otherwise: Set to False to ensure FULL_ACTORS_CREDIT info is not added.

  GET_FULL_MPAA  = True ;  //Download Certification provider page for retreive the MPAA info. Otherwise only the info of the principal movie page.
Quote
    //Get  to "Cast" ~actors~ ctActors
    //Go Cast list
   If (GET_MAINPAGE_ACTORS_CREDIT) Then Begin   
       curPos:=Pos('<table class="cast_list">',HTML);                                     //Strings start which opens the block content data. WEB_SPECIFIC
       curPos:=curPos+Length('<table class="cast_list">');                                //Strings end which opens the block content data.  WEB_SPECIFIC
       //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~
       endPos:=PosFrom('</div>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
       ItemList:=Copy(HTML,curPos,endPos-curPos);
       //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');List+'||');
       curPos:=Pos('<td class="primary_photo">',ItemList)                               //String which opens the subList data. WEB_SPECIFIC
       index:=1;
       While curPos>0 Do Begin
          If (index>PEOPLE_LIMIT) Then break;     //Limited depassed (Remember index begin in 0).
          //Get PersonURL (Always must it has)
          PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF; //Strings which opens/closes the data. WEB_SPECIFIC
          LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
          //Get Name (Always must it has)
          Name:=TextBetWeen(ItemList,'<td>','</td>',false,curPos);                             //Strings which opens/closes the data. WEB_SPECIFIC
          LogMessage('      Parse Results Name:'+Name+'||');
          //Get Role NO: PVD don't save Role in crew people
          //Get Role
          Role:=TextBetWeen(ItemList,'<td class="character">','</td>',false,curPos);                      //Strings which opens/closes the data. WEB_SPECIFIC         
          LogMessage('      Parse Results Role:'+Role+'||');
        Role:=StringReplace(Role,'       (',' (',True,False,True);
        Role:=StringReplace(Role,'(uncredited)                    ','(uncredited) • ',True,False,True);
        Role:=StringReplace(Role,')                    ',') ',True,False,True);
        Role:=StringReplace(Role,' /              ',' - ',True,False,True);
        Role:=StringReplace(Role,'         / ...                    ',' • ',True,False,True);
        Role:=StringReplace(Role,'                  ',' • ',True,False,True);      
        Role:=StringReplace(Role,'      (uncredited)',' (uncredited)',True,False,True);
          LogMessage('      Parse Results Role_1:'+Role+'||');      
          AddMoviePerson(Name,'',Role,PersonURL,ctActors);       
          //AddMoviePerson(Name,'','',PersonURL,ctActors);
          LogMessage('      Get results cast:#'+IntToStr(index)+'|'+Name+'|'+Role+'|'+PersonURL+'||ctActors');                                               
          curPos:=PosFrom('<td class="primary_photo">',ItemList,curPos);                                  //String which opens the Web Result item List data. WEB_SPECIFIC
          index:=index+1;
       End;
    End;

Full Cast and Crew section

Quote
    //Get  to "Cast" ~actors~ ctActors
    //Go Cast list
   If( GET_FULL_ACTORS_CREDIT) Then Begin
    curPos:=Pos('<table class="cast_list">',HTML);                                     //Strings start which opens the block content data. WEB_SPECIFIC
    curPos:=curPos+Length('<table class="cast_list">');                                //Strings end which opens the block content data.  WEB_SPECIFIC
    //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~
    endPos:=PosFrom('</div>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
    ItemList:=Copy(HTML,curPos,endPos-curPos);
    //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');
    curPos:=Pos('<td class="primary_photo">',ItemList)                               //String which opens the subList data. WEB_SPECIFIC
    index:=1;
    While curPos>0 Do Begin
        If (index>PEOPLE_LIMIT) Then break;     //Limited depassed (Remember index begin in 0).
        //Get PersonURL (Always must it has)
        PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF; //Strings which opens/closes the data. WEB_SPECIFIC
        LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
        //Get Name (Always must it has)
        Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                             //Strings which opens/closes the data. WEB_SPECIFIC      
        LogMessage('      Parse Results Name:'+Name+'||');
        //Get Role
        Role:=TextBetWeen(ItemList,'<td class="character">','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC         
        LogMessage('      Parse Results Role:'+Role+'||');
      Role:=StringReplace(Role,'       (',' (',True,False,True);
      Role:=StringReplace(Role,'(uncredited)                    ','(uncredited) • ',True,False,True);
      Role:=StringReplace(Role,')                    ',') ',True,False,True);
      Role:=StringReplace(Role,' /              ',' - ',True,False,True);
      Role:=StringReplace(Role,'         / ...                    ',' • ',True,False,True);
      Role:=StringReplace(Role,'                  ',' • ',True,False,True);      
      Role:=StringReplace(Role,'      (uncredited)',' (uncredited)',True,False,True);
      LogMessage('      Parse Results Role_1:'+Role+'||');      
        AddMoviePerson(Name,'',Role,PersonURL,ctActors);
        LogMessage('      Get results cast:#'+IntToStr(index)+'|'+Name+'|'+Role+'|'+PersonURL+'||ctActors');                                               
        curPos:=PosFrom('<td class="primary_photo">',ItemList,curPos);                                  //String which opens the Web Result item List data. WEB_SPECIFIC
        index:=index+1;
    End; End;
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on October 20, 2018, 07:52:00 pm
Attached temporal Acumulative Scripts Patch date:2018.10.20 (to copy in PVD folder for overwriting the existing script files).

Changelog: (All posted alerts treated but the last GET_FULL_CREDIT=False)

IMDB_[EN][HTTPS].psf and IMDB_People_[EN][HTTPS].psf:
V 1.2.0.3 (20/10/2018) Improvement of search stored URL. Better script flow control. Only get FULL_AKA in MediaType='Movie'. More Ivek23 corrections.

FilmAffinity_[xx][HTTPS].psf
V 4.0.0.6 (20/10/2018) VVV: Improvement of search stored URL. Added the custom field FilmAffinity_Rating. Better script flow control.

Quote
It is necessary to check " // Get Cast." GET_FULL_CREDIT = False only the info of the main movie page. " code in Function ParsePage_IMDBMovieBASE section
I agree but there is enougth info of the directors, authors,  composers, etc in the principal page for the normal user (for instance for me ;D )...
but now I have a very little amont of free time. Can anybody help with the script?  :)


 
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on October 21, 2018, 10:08:29 am
The same acumulative patch but with LogMessages improvements (only for the scripts developers: better tabulation, texts with the Custom Field name, etc)
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 21, 2018, 11:59:25 am
Quote
    //Date ~Updated~ (choose simple or verbose version)
        Date:=DateToStr(CurrentDateTime);
        //AddFieldValueXML('viewdate',Date); //Only date, don't admit time-. Set Seen value at the same time.
        //AddFieldValueXML('moddate',Date+' '+TimeToStr(CurrentDateTime)); //Block the dB saving   
        ExplodeString(Date,DateParts,'-');
        Date:=DateParts[2]+'.'+ DateParts[1]+'.'+DateParts[0]; 
        Date:=StringReplace(Date,'01.','1.',True,True,False);
      Date:=StringReplace(Date,'02.','2.',True,True,False);
      Date:=StringReplace(Date,'03.','3.',True,True,False);
      Date:=StringReplace(Date,'04.','4.',True,True,False);
      Date:=StringReplace(Date,'05.','5.',True,True,False);
      Date:=StringReplace(Date,'06.','6.',True,True,False);
      Date:=StringReplace(Date,'07.','7.',True,True,False);
      Date:=StringReplace(Date,'08.','8.',True,True,False);
      Date:=StringReplace(Date,'09.','9.',True,True,False);   
      
        //AddCustomFieldValueByName('Updated',Date); // Simple
        //AddCustomFieldValueByName('Updated',Date+' at '+TimeToStr(CurrentDateTime)); // Verbose
        AddCustomFieldValueByName('Updated',Date+' at '+TimeToStr(CurrentDateTime)+' • '+SCRIPT_NAME+' '+SCRIPT_VERSION);  // Annoying

IMDB_ [EN] [HTTPS] script

A small error in a piece of code for Date ~ Updated ~ (choose simple or verbose version) .

Quote
    //Date ~Updated~ (choose simple or verbose version)
        Date:=DateToStr(CurrentDateTime);
        //AddFieldValueXML('viewdate',Date); //Only date, don't admit time-. Set Seen value at the same time.
        //AddFieldValueXML('moddate',Date+' '+TimeToStr(CurrentDateTime)); //Block the dB saving   
        ExplodeString(Date,DateParts,'-');
        Date:=DateParts[2]+'.'+ DateParts[1]+'.'+DateParts[0]; 
      Date:=StringReplace(Date,'02.','2.',True,True,False);
      Date:=StringReplace(Date,'03.','3.',True,True,False);
      Date:=StringReplace(Date,'04.','4.',True,True,False);
      Date:=StringReplace(Date,'05.','5.',True,True,False);
      Date:=StringReplace(Date,'06.','6.',True,True,False);
      Date:=StringReplace(Date,'07.','7.',True,True,False);
      Date:=StringReplace(Date,'08.','8.',True,True,False);
      Date:=StringReplace(Date,'09.','9.',True,True,False);      
        //AddCustomFieldValueByName('Updated',Date); // Simple
        //AddCustomFieldValueByName('Updated',Date+' at '+TimeToStr(CurrentDateTime)); // Verbose
        AddCustomFieldValueByName('Updated',Date+' at '+TimeToStr(CurrentDateTime)+' • '+SCRIPT_NAME+' '+SCRIPT_VERSION);  // Annoying

Missing this in a piece of code for Date ~ Updated ~ (choose simple or verbose version) .
        Date:=StringReplace(Date,'01.','1.',True,True,False);     
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: abubin on October 21, 2018, 01:47:36 pm
any chance of getting this working with Firebird database that is hosted on a remote server? My PVD is running as such but tested this mod version and it could not load the database using "Connect to server...".

Thanks for the mod effort! Appreciate it.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on October 22, 2018, 08:39:30 am
Quote
working with Firebird database
I think that in the MOD version there are only two modification that may to influence this functionality: it runs in portable mode and there is a dll update.
I can propose you some test (important, first make a backup of your actual instalation):

1) Replace the Scripts and Plugings folders of your current installation with the MOD folder but run it in normal mode (NOT with Portable.bat)
 With this you have a MOD working copy over your standart installation.
2) If it works, copy the MOD dll's to the current installation. If it works, I suspect that the portable mode does not allow the connection with the Firefox server, if it does not work I suspect that the problem is the new dll.

Let us know your results.

(see "Dirty" own PVD installation/collection adaptation for advanced users" (http://www.videodb.info/forum_en/index.php/topic,4134.msg20627.html#msg20627) for more information)
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: jondak on October 25, 2018, 06:31:19 pm
Hello,

are the movie connections parsed by the MOD version ?

example:
https://www.imdb.com/title/tt0071275/movieconnections/?tab=mc&ref_=tt_trv_cnn

Cheers
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 25, 2018, 08:59:59 pm
Hello,

are  parsed by the MOD version ?

example:
https://www.imdb.com/title/tt0071275/movieconnections/?tab=mc&ref_=tt_trv_cnn

Cheers

In the primary IMDB_ [EN] [HTTPS] script definitely will not be added more new information, including not the movie connections or any other possible information.

Notice of such a decision:
 
This is what can now be downloaded with the script, of course, it is also the most information that the script can download. This is appropriate but not in too long, and it is safe, that this information can still be downloaded by this script. Already now it's time to download information pretty much long, everything depends on how much information is written on web pages, which the script must then download. Thus, the transmission time of the information is still fast and expansive in the transmission of information, thus preventing the possibility of the PVD program being accidentally frozen when transferring information.

There is, however, the next option to be in the secondary IMDB_ [EN] [HTTPS] script adds more new information related to other information, including the movie connections.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on October 26, 2018, 08:59:04 pm
Attached temporal Acumulative Scripts Patch date:2018.10.26 (to copy in PVD folder for overwriting the existing files).
ATTENTION: For recover Toolbar it overwrite the 'pvdconf.ini' file and Tabs folder.

Changelog:
                Corrected "Date:=StringReplace(Date,'01.','1.',True,True,False);"
                Now retrive directors, authors,  composers, etc from the principal page
                Recover the Toolbar with the indispensable buttons (Delete posters, clear data,etc)

Note: This is a release candidate.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 28, 2018, 01:41:31 pm
IMDB_[EN][HTTPS] (Connections) script

Hello,

are  parsed by the MOD version ?

example:
https://www.imdb.com/title/tt0071275/movieconnections/?tab=mc&ref_=tt_trv_cnn

Cheers

In the primary IMDB_ [EN] [HTTPS] script definitely will not be added more new information, including not the movie connections or any other possible information.

Notice of such a decision:
 
This is what can now be downloaded with the script, of course, it is also the most information that the script can download. This is appropriate but not in too long, and it is safe, that this information can still be downloaded by this script. Already now it's time to download information pretty much long, everything depends on how much information is written on web pages, which the script must then download. Thus, the transmission time of the information is still fast and expansive in the transmission of information, thus preventing the possibility of the PVD program being accidentally frozen when transferring information.

There is, however, the next option to be in the secondary IMDB_ [EN] [HTTPS] script adds more new information related to other information, including the movie connections.

I've edited the MOD version of IMDB_ [EN] [HTTPS] script, which is now called IMDB_ [EN] [HTTPS] (Connections) script to transfer the Movie Connections information to the Normal Connections PVD field.

An additional code has also been added to transfer the category, which is allowed to record
url titles in the Connections custom box for links to the Connections website. In the Connections custom field, your data record will look like this,
Quote
Followed by (2)   Version of (58)   Spin-off (1)   References (6)   Referenced in (27)   Features (1)   Featured in (1)   Spoofs (1)   Spoofed in (2)
it will also have a visible option in the record, clicking on it, in order to open the Internet window of this record in the web browser.

In case of only such a record,
Quote
Spin-off (1)
there will be no url link to the Connections website.

IMDB_ [EN] [HTTPS] (Connections) script is attached.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 28, 2018, 02:23:04 pm
IMDB_ [EN] [HTTPS] (Connections) script

Notification:


There is, however, an option, that the script will be transmitting information for a long time due to the large number of Movie Title. This is the case of Casablanca movie (http://),
Quote
https://www.imdb.com/title/tt0034583/movieconnections
with more than 1500 records.

If anyone has a suggestion for which category of records should be the transfer of information, he will be welcome.

The list is as follows:
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on October 29, 2018, 07:49:50 pm
And get only Movies not TV Episodes? Is it possible?
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 29, 2018, 08:43:14 pm
And get only Movies not TV Episodes? Is it possible?

I will see what to do with this case. I do not know the result at the moment, because when I created the Imdb Movie (connections) script,  nobody not mentioned this problem already at that time. At that time, I did not tested it, and then repaired it or already found a way, how to find the option to download all records or records without TV Episodes.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on October 29, 2018, 10:00:32 pm
Attached temporal Acumulative Scripts Patch date:2018.10.29 (to copy in PVD folder for overwriting the existing files).
ATTENTION: For recover Toolbar it overwrite the 'pvdconf.ini' file and Tabs folder.

Changelog:
        IMDB_[EN][HTTPS]+IMDB_People_[EN][HTTPS]
            V 1.3.0.0 (28/10/2018) New parameter USE-SAVED-PVDCONFIG for avoid download page if the Overwrite Options of the Script in
                                        the pvdconf.ini are 'Do Nothing'. Remember that PDV only save the pvdconf.ini in exit.

Note: This is a release candidate.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on October 30, 2018, 07:35:27 am
And get only Movies not TV Episodes? Is it possible?

I will see what to do with this case. I do not know the result at the moment, because when I created the Imdb Movie (connections) script,  nobody not mentioned this problem already at that time. At that time, I did not tested it, and then repaired it or already found a way, how to find the option to download all records or records without TV Episodes.

It is not possible to transfer data without TV Episodes records, because the problem is in the URLs that are written for TV Episodes records.

There is a possibility that certain categories, for example version_of and similar categories, which contain TV Episodes records, are completely blocked for the transmission of information.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on November 01, 2018, 10:39:17 am
Changelog:
Last version always in the first post
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on November 01, 2018, 04:02:50 pm
IMDB_[EN][HTTPS] script

Quickly found in Script version, a small beauty error in version number where one zero is missing.

Before
Quote
SCRIPT_VERSION  = '1.3.0.';
Correct
Quote
SCRIPT_VERSION  = '1.3.0.0';
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Miguelh1020 on November 11, 2018, 12:24:47 am
Hey guys. Thanks for everything. There are a couple of things I couldn't work out.
I can't get all the tags from a movie. The IMDB[EN] script only downloads 4-5 ( PVD calls them Tags but IMDB calls them Plot Keywords).

The other problem is with Series. How do I do make series appear as seasons and episodes? When I use IMDB[EN] I only get a main page and not the episodes and I don't know how to add any.
Also it seems it doesn't import Directors, producers, writers or composers.

Thanks for all your work!
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on November 11, 2018, 08:22:25 am
Hey guys. Thanks for everything.
.
.
.
Thanks for all your work!

Welcome and thanks.

I can't get all the tags from a movie. The IMDB[EN] script only downloads 4-5 ( PVD calls them Tags but IMDB calls them Plot Keywords).

Tags are in fact really Plot Keywords. Tags is a PVD field for downloading Plot Keywords information. For Plot Keywords it will be possible to create a standalone IMDB_ [EN] [HTTPS] (Plot Keywords) script. In the basic IMDB_ [EN] [HTTPS] script, Plot Keywords is only downloaded from the primary IMDB site, but there are only as many tags as you have already mentioned.

The other problem is with Series. How do I do make series appear as seasons and episodes? When I use IMDB[EN] I only get a main page and not the episodes and I don't know how to add any.

As for the series and the episode, it is necessary to create a brand new IMDB_ [EN] [HTTPS] script, which would only transmit information for the series. Here, then, there is a problem, because a lot of code changes are required in the script, among which the beginnings of the series are already recorded in the beginning of the series as in movies, and it is likely that something would be found that would cause problems with the good functioning of the script.

Perhaps a script will be made in the future, but not currently, because the basic IMDB_ [EN] [HTTPS] script needs to be stored to the details for complete and correct operation. The same goes for the PVD MOD program.

Also it seems it doesn't import Directors, producers, writers or composers.

As far as this is concerned, it does not transfer with the basic IMDB_ [EN] [HTTPS] script, because it is completely different source code of the page for the series for the previously mentioned categories of persons.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: jondak on November 17, 2018, 08:44:27 pm
Found a bug in the last version in the IMDB_ [EN] [HTTPS] script in both 9.9.2.1 and 1.0.2.7

The error comes if the Script configuration box is ticked: Download "Also know as' provider page for retrieve the info...
If the box is not checked the script works and saves the info.

I atached the bug report.

I got the error while i tryed to update the following movies:

Calendar Girl (2011) https://www.imdb.com/title/tt1611816/
The Brazen Bull (2010) https://www.imdb.com/title/tt1415284/

The script worked till 16.11.2018, probably something changed to imdb site
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on November 18, 2018, 06:29:50 pm
Found a bug in the last version in the IMDB_ [EN] [HTTPS] script in both 9.9.2.1 and 1.0.2.7

The error comes if the Script configuration box is ticked: Download "Also know as' provider page for retrieve the info...
If the box is not checked the script works and saves the info.

I atached the bug report.

I got the error while i tryed to update the following movies:

Calendar Girl (2011) https://www.imdb.com/title/tt1611816/
The Brazen Bull (2010) https://www.imdb.com/title/tt1415284/

The script worked till 16.11.2018, probably something changed to imdb site

Yes, a small change in the source code of the page.

This is to fix it and it's working again.
Quote
Function ParsePage_IMDBMovieAKA(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit;
    //Retrieve: ~aka~ "Also Known As"
  Var
    curPos,endPos,index:Integer;
    ItemValue,ItemList:String;
    ItemArray: TWideArray;
  Begin
    LogMessage('Function ParsePage_IMDBMovieAKA BEGIN=====================||');
    Result:=prFinished;  //It will change to prError if any big problem with exit;
    //Get ~aka~ (several values in a line break separated list) Be careful don't repeat
    if (Pos('It looks like we don'+Chr(39)+'t have any AKAs for this title yet.',HTML)>0) then Exit;
    //Go to "Also Known As"
   //curPos:=Pos('<h4 class="li_group">Also Known As (AKA)',HTML);
    curPos:=Pos('class="ipl-list-title">Also Known As (AKA)',HTML);                      //Strings start which opens the block content data. WEB_SPECIFIC
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: jondak on November 22, 2018, 07:56:17 am
Thank you.

The fix worked for the titles i linked. But it seems not all titles are made the same  :-\

Dai juk hei kek (2012) https://www.imdb.com/title/tt2266938/ has a crash also.

Maybe is from the chirilic alphabet on the AKA page?

Cheers.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on November 22, 2018, 07:57:49 pm
Thank you.

The fix worked for the titles i linked. But it seems not all titles are made the same  :-\

Thanks.

Dai juk hei kek (2012) https://www.imdb.com/title/tt2266938/ has a crash also.

Maybe is from the chirilic alphabet on the AKA page?

Cheers.

No, it was not what you thought, but was missing the name for the country. This should now solve the problem, at least it worked for me.

Quote
Function ParsePage_IMDBMovieAKA(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError;
    //Retrieve: ~aka~ "Also Known As"
  Var
    curPos,endPos,index:Integer;
    ItemValue,ItemList:String;
    ItemArray: TWideArray;
  Begin
    LogMessage('Function ParsePage_IMDBMovieAKA BEGIN=====================||');
    //Get ~aka~ (several values in a line break separated list) Be careful don't repeat
    Result:=prFinished;
    if (Pos('It looks like we don'+Chr(39)+'t have any AKAs for this title yet.',HTML)>0) then Exit;
    //Go to "Also Known As"
   //curPos:=Pos('<h4 class="li_group">Also Known As (AKA)',HTML);
   curPos:=Pos('class="ipl-list-title">Also Known As (AKA)',HTML);                      //Strings start which opens the block content data. WEB_SPECIFIC
    curPos:=PosFrom('</h4>',HTML,curPos);                                               //Strings end which opens the block content data.  WEB_SPECIFIC
    curPos:=curPos+Length('</h4>');                                                    //Strings end which opens the block content data.  WEB_SPECIFIC
    //Get all "raw" list aka for create good values separators
    endPos:=PosFrom('</table>',HTML,curPos);                                            //Strings which opens/closes the data. WEB_SPECIFIC
    ItemList:=Copy(HTML,curPos,endPos-curPos);
   ItemList:=StringReplace(ItemList,'<td class="aka-item__name--empty"></td>','(Empty)|',True,True,False);             //Fill the empty values (in countries) con in order to asure that ExplodeString get a value. WEB_SPECIFIC
    ItemList:=StringReplace(ItemList,'<td></td>','(Empty)|',True,True,False);             //Fill the empty values (in countries) con in order to asure that ExplodeString get a value. WEB_SPECIFIC   
    ItemList:=StringReplace(ItemList,'</td>','|',True,True,False);                      //WEB_SPECIFIC
    //Get all aka list (NOT necesary in raw because: Several couple of valors (Country/title) but the first is the original title
    ItemList:=RemoveTags(ItemList, False);
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on November 23, 2018, 08:45:49 am
And get only Movies not TV Episodes? Is it possible?

I will see what to do with this case. I do not know the result at the moment, because when I created the Imdb Movie (connections) script,  nobody not mentioned this problem already at that time. At that time, I did not tested it, and then repaired it or already found a way, how to find the option to download all records or records without TV Episodes.

Yes, I found a way to remove the episode from the records. Here is a filter that removes everything except movies records.
Example code:
Quote
Function ParsePage_IMDBMovieCONNECTIONS(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit
    //Retrieve: ~links~
  Var
    //curPos:Integer;
    curPos,endPos,index:Integer;
   debug_pos1:Integer;
    //ItemValue,ItemList:String;
    OrigTitle,Title,Category3,Category2,Category1,Category,URL,Year:String;
  Begin
.
.
.
   //Movie Connections Category for the Connections Title of Category
    //Get "Connections" info
    curPos:=Pos('<h1 class="header">Connections</h1>',HTML);
    if (curPos=0) then Exit;
    EndPos:=curPos;
   //Title of Movies for the Connections Category into the PVD database
    //Get "follows" info
   curPos:=PosFrom('<a id="follows" name="follows"></a>',HTML,EndPos);
   If 0<curPos Then Begin
      EndPos:=curPos;
         curPos:=PosFrom('<h4 class="li_group">',HTML,curPos);
         endPos :=PosFrom('</h4>',HTML,curPos);
         Category:=TextBetween(HTML,'<h4 class="li_group">','</h4>',false,curPos);   
         If Category <> '' then LogMessage('      Get result Category:'+Category+'||');
      //end;
      curPos:=PosFrom('" ><a href="/title/',HTML,curPos);
      index:=1;
      while (curPos > 0) AND (curPos < PosFrom('<h',HTML,endPos)) do begin
         endPos:=PosFrom('">',HTML,curPos);
         URL:='http://www.imdb.com/title/' + TextBetween(HTML,'" ><a href="/title/','">',false,curPos)+'/';
         If URL <> '' then LogMessage('      Get result URL:'+URL+'||');
         Title:=TextBetween(HTML,'">','</',false,curPos);   
         If Title <> '' then LogMessage('      Get result Title:'+Title+'||');
         Year   :=TextBetween(HTML,'(',')',false,curPos);
         If Year <> '' then LogMessage('      Get result Year:'+Year+'||');
            curPos:=PosFrom('</a>',HTML,endPos)+5;
            endPos:=PosFrom('</div>',HTML,curPos);      
            OrigTitle:=RemoveTags(Copy(HTML,curPos,endPos-curPos),false);
            LogMessage('      Get result OrigTitle1:'+OrigTitle+'||');
            OrigTitle:=StringReplace(OrigTitle,')(',') (',True,False,True);
            debug_pos1:=Pos('    ',OrigTitle);
            If debug_pos1 > 0 then OrigTitle:=Copy(OrigTitle,0,debug_pos1-1);         
            OrigTitle:=StringReplace(OrigTitle,' (TV Movie)','',True,False,True);
            If OrigTitle <> '' then LogMessage('      Get result OrigTitle:'+OrigTitle+'||');      
   
         //If Pos(': ',Title) < 1 then
         If Pos('(TV Episode)',OrigTitle) OR Pos('(TV Series)',OrigTitle) OR Pos('(TV Mini-Series)',OrigTitle) OR Pos('(Short)',OrigTitle) OR Pos('(TV Short)',OrigTitle) OR Pos('(TV Special)',OrigTitle) OR Pos('(Video Game)',OrigTitle) OR Pos('(Video)',OrigTitle) < 1 then 
         //AddConnection(Title,'',Category,URL,Year);               
         AddConnection(Title,Title+' '+OrigTitle,Category,URL,Year);      
         LogMessage('      Get results follows:#'+IntToStr(index)+'|'+Title+'|'+Year+'|'+Category+'||');
         curPos := PosFrom('" ><a href="/title/',HTML,curPos);   
         index:=index+1;
      End;
   End Else
   curPos:=endPos;

I added the OrigTitle code, which is essential for filtering records at all and does not change. In the Connection field, the notes will also have a year (for example, For Your Eyes Only (1981)), which is written only in the Original Title field, but not in the Titile field in the PVD database. This option is added so that you often see multiple identical titles of the title of a movie, but the years of the movie outcome are different. This is removed at that time (movie year) when you update the movie record and the Original Title checkbox is confirmed.

I'm currently testing Imdb Movie (connections) script and I'm also looking for options for any additional filters for filtering records.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on November 23, 2018, 10:28:43 am
IMDB_ [EN] [HTTPS] (Plot Keywords) script

I can't get all the tags from a movie. The IMDB[EN] script only downloads 4-5 ( PVD calls them Tags but IMDB calls them Plot Keywords).

Tags are in fact really Plot Keywords. Tags is a PVD field for downloading Plot Keywords information. For Plot Keywords it will be possible to create a standalone IMDB_ [EN] [HTTPS] (Plot Keywords) script. In the basic IMDB_ [EN] [HTTPS] script, Plot Keywords is only downloaded from the primary IMDB site, but there are only as many tags as you have already mentioned.

An IMDB_ [EN] [HTTPS] (Plot Keywords) script for full download of Plot Keywords information was made.

IMDB_ [EN] [HTTPS] (Plot Keywords) script is attached.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on November 23, 2018, 09:07:10 pm
Attached temporal Acumulative Scripts Patch date: 2018.11.23 (to copy in PVD folder for overwriting the existing files).

Changelog:
        IMDB_[EN][HTTPS]
        V 1.3.1.0 (22/11/2018) Ivek23: ParsePage_IMDBMovieAKA small change in the source code and was missing the name for the country.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on November 25, 2018, 10:52:55 am
Attached temporal Acumulative Scripts Patch date: 2018.11.23 (to copy in PVD folder for overwriting the existing files).

Changelog:
        IMDB_[EN][HTTPS]
        V 1.3.1.0 (22/11/2018) Ivek23: ParsePage_IMDBMovieAKA small change in the source code and was missing the name for the country.

A minor error in the code ParsePage_IMDBMovieAKA code where the letter "c" is missing
Quote
    urPos:=Pos('class="ipl-list-title">Also Known As (AKA)',HTML);                      //Strings start which opens the block content data. WEB_SPECIFIC

It is correct:
Quote
   curPos:=Pos('class="ipl-list-title">Also Known As (AKA)',HTML);                      //Strings start which opens the block content data. WEB_SPECIFIC
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on November 25, 2018, 08:12:13 pm
Damn! I had corrected that error that also prevents the compilation ... but I had to upload the bad file :-[  . Thanks Ivek23.
Now I think it's fine.

Attached temporal Acumulative Scripts Patch date: 2018.11.25 (to copy in PVD folder for overwriting the existing files).

Changelog:
        IMDB_[EN][HTTPS]
        V 1.3.1.0 (22/11/2018) Ivek23: ParsePage_IMDBMovieAKA small change in the source code and was missing the name for the country.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on November 28, 2018, 08:01:19 am
IMDB_[EN][HTTPS] script

This need to be fixed. Now records in ShowMessage are not in English.


Quote
Function DownloadPage(URL:AnsiString):String; //BlockOpen
//Returns the URL page text. If error returns empty string
  Var
.
.
.
         If i=INTERNET_TEST_ITERATIONS Then Begin
            //if 2=MessageBox('Demasiados intentos fallidos de conexión a internet.'+Chr(13)+ 'Reintentar o             //if 2=MessageBox('Demasiados intentos fallidos de conexión a internet.'+Chr(13)+ 'Reintentar o Cancelar?',SCRIPT_NAME,5) then begin
         if 2=MessageBox('Too many faulty attempts to internet connection.'+Chr(13)+ 'Retry or Cancel?',SCRIPT_NAME,5) then begin
               LogMessage(Chr(9)+Chr(9)+'Function DownloadPage END with NOT INTERNET connection ===============|');
               Result:='';
               Exit;
            End;
            i:=0;
         End;
.
.
.
    if (Pos('405 Method not allowed',Result)>0) then begin  //Banned IP. WEB_SPECIFIC.
        //If BYPASS_SILENT Then ShowMessage('La URL tiene problemas de HTTP metodo (405 Method not allowed).'+Chr(13)+'Ir a la página del proveedor para'+Chr(13)+'encontrar la página buena',SCRIPT_NAME);
        If BYPASS_SILENT Then ShowMessage('The URL has HTTP method problems (405 Method not allowed).'+Chr(13)+'Go to the provider web in order to in order to'+Chr(13)+'find the good page',SCRIPT_NAME);      
        LogMessage(Chr(9)+Chr(9)+'      405 Method not allowed|');
        Result:=''; //Fatal error
    End;
    if (Pos('Too many request',Result)>0) then begin  //Banned IP. WEB_SPECIFIC.
        //If BYPASS_SILENT Then ShowMessage('El proveedor ha prohibido su IP (Demasiadas peticiones).'+Chr(13)+'Ir a la página del proveedor y resolver el captcha para'+Chr(13)+'demostrar que no es un robot',SCRIPT_NAME);
        If BYPASS_SILENT Then ShowMessage('The provider has banned your IP (Too many request).'+Chr(13)+'Go to the provider web and resolve the captcha in order to'+Chr(13)+'demostrate you are not a robot',SCRIPT_NAME);      
        LogMessage(Chr(9)+Chr(9)+'      Banned IP|');
        Result:=''; //Fatal error
    End;
    LogMessage(Chr(9)+Chr(9)+'Function DownloadPage END======================|');
    exit;
  End; //BlockClose
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on November 28, 2018, 09:02:33 am
IMDB_ [EN] [HTTPS] (Plot Keywords) script

IMDB_ [EN] [HTTPS] (Plot Keywords) script now has the option to download the full number of Plot Keywords records. Also, the possibility of a limited number of Plot Keywords records is added, because in some cases a very large number of records is also available for a specific movie (the numbers also range over 400 or even 500 or more records). The limit is set to the number of 100 records in the PVD Tags field. IMDB_ [EN] [HTTPS] (Plot Keywords) script still transmits the full number of records, after which the filter allows 100 records in the PVD Tags field. Thus, IMDB_ [EN] [HTTPS] (Plot Keywords) script in some cases requires a lot of time to add records to the PVD database. I am still looking for a way or solution so that only a limited number of records (100) can be transmitted by IMDB_ [EN] [HTTPS] (Plot Keywords) script and written in the PVD Tags field and therefore it would take less time to add records to the PVD database .

Quote
  GET_FULL_PLOTKEYWORDS  = True ;  //Download Plot Keywords provider page for retreive the Plot Keywords info. Otherwise only the info of the principal movie page
  GET_PLOTKEYWORDS_LIMIT  = True ;  //Download Full of number of tags retrieved. Otherwise True for the only the Limit of number of tags retrieved.      * Get Limit of number of tags retrieved.
//  GET_PLOTKEYWORDS_LIMIT  = False ;  //Download Full of number of tags retrieved. Otherwise True for the only the Limit of number of tags retrieved.   ** Get Full of number of tags retrieved.
  PLOTKEYWORDS_LIMIT  = 100;    //Limit of number of tags retrieved.

In IMDB_ [EN] [HTTPS] (Plot Keywords) script, you can manually modify the way you transfer your records by blocking the current open setting and unblocking the currently blocked setting. With this change, the script will download the full number of records.

The new version of IMDB_ [EN] [HTTPS] (Plot Keywords) script is attached.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on December 01, 2018, 03:34:45 pm
IMDB_ [EN] [HTTPS] (Plot Keywords) script

Now there is the final version of IMDB [EN] [HTTPS] (Plot Keywords) script for use.

Changes to the final version of IMDB [EN] [HTTPS] (Plot Keywords) script:

Quote
  GET_FULL_PLOTKEYWORDS  = True ;  //Download Plot Keywords provider page for retreive the Plot Keywords info. Otherwise only the info of the principal movie page
  GET_PLOTKEYWORDS_LIMIT  = True ;  //Download Full of number of tags retrieved. Otherwise True for the only the Limit of number of tags retrieved.      * Get Limit of number of tags retrieved.
//  GET_PLOTKEYWORDS_LIMIT  = False ;  //Download Full of number of tags retrieved. Otherwise True for the only the Limit of number of tags retrieved.   ** Get Full of number of tags retrieved.
  PLOTKEYWORDS_LIMIT  = 100;    //Limit of number of tags retrieved.

The change is now described correctly and now looks like this:
Quote
  GET_FULL_PLOTKEYWORDS  = True ;  //Download Plot Keywords provider page for retreive the Plot Keywords info. Otherwise only the info of the principal movie page
  GET_PLOTKEYWORDS_LIMIT  = True ;    //Download Limit of number of tags retrieved.   * Get True for Limit of number of tags retrieved.
//  GET_PLOTKEYWORDS_LIMIT  = False ;    //Download Full of number of tags retrieved.  ** Get False for Full of number of tags retrieved.
  PLOTKEYWORDS_LIMIT  = 100;    //Limit of number of tags retrieved.
The download limit is set to 100 tags, but the script transfers 101 tags because the index starts to count at 0 digits.
In IMDB_ [EN] [HTTPS] (Plot Keywords) script, you can manually modify the way you transfer your records by blocking the current open setting and unblocking the currently blocked setting. With this change, the script will download the full number of records.

Added Plot Keywords to download info on the main movie page.

Now, the script works to limit the number of records faster, because I found a way that is faster than the previous one and only transfers as many records as the actual number is written in PLOTKEYWORDS_LIMIT options.

Quote
Function TextBetWeenFirst(_HTML:string;_ABegin,_AEnd:String):String; //BlockOpen
    //Returns the first extract part of a given _HTML between _Abegin and _AEnd and removes all tags from the result substring.
    //In order to avoid set the Pos variable in TextBetween
  Var
    _Pos:Integer;
  Begin
    _Pos:=0;
    Result:=TextBetWeen(_HTML,_ABegin,_AEnd,false,_Pos);
    Exit;   
End; //BlockClose 
function RemoveTagsEx(AText:String):String; //BlockOpen
Var
   B,E:Integer;
Begin
   Result:=AText;
   B:=PosFrom('    <div class="did-',Result,18);
   E:=PosFrom('      </span>',Result,B);
   While (B>0) AND (B<E) Do Begin
      Delete(Result,B,E-B+18);
      B:=Pos('    <div class="did-',Result);
      E:=Pos('      </span>',Result);
   End;
End; //BlockClose

Function ParsePage_BingSearch(HTML:String):Cardinal; //BlockOpen

This part of the code is also needed, otherwise the script does not work.

Also added is the Plot Keywords custom field in which the URL is transferred to the website and how much is the full number of Plot Keywords records for a specific movie title on the Plot Keywords web page. You can Plot Keywords custom field settings found in Possibles improving Custom Fields working in MOD version topic. (http://www.videodb.info/forum_en/index.php/topic,4141.0.html)

The final version of IMDB [EN] [HTTPS] (Plot Keywords) script is attached.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on December 02, 2018, 10:41:08 am
Well done, Ivek32.
You really master the script working flow of the PVD MOD.
I'm going to integrate the retreive of Plotkeywords in the base script of IMDB (although 100 seem to me many) and Connections when you tell me it is ready.
Now I do not have much time, but it is first in my ToDo list.

Note: If Nostra wakes up and simply compiles the actual PVD with the libraries for HTTPS, we would have powerful scripts that could be quickly adapted to the oficial version.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on December 02, 2018, 04:08:43 pm
Well done, Ivek32.
You really master the script working flow of the PVD MOD.
I'm going to integrate the retreive of Plotkeywords in the base script of IMDB (although 100 seem to me many) ...

Thank you.
 
I agree that there are 100 records too, so my suggestion is up to 50 records. I tend to be more inclined to a more appropriate number, that is, only 25 records in the database, what is the best variant that occupies a bit of space, especially when it is important in the record, which has a larger number of words in one record.

Note:
Quote
Function TextBetWeenFirst(_HTML:string;_ABegin,_AEnd:String):String; //BlockOpen
    //Returns the first extract part of a given _HTML between _Abegin and _AEnd and removes all tags from the result substring.
    //In order to avoid set the Pos variable in TextBetween
  Var
    _Pos:Integer;
  Begin
    _Pos:=0;
    Result:=TextBetWeen(_HTML,_ABegin,_AEnd,false,_Pos);
    Exit;   
End; //BlockClose 
function RemoveTagsEx(AText:String):String; //BlockOpen
Var
   B,E:Integer;
Begin
   Result:=AText;
   B:=PosFrom('    <div class="did-',Result,18);
   E:=PosFrom('      </span>',Result,B);
   While (B>0) AND (B<E) Do Begin
      Delete(Result,B,E-B+18);
      B:=Pos('    <div class="did-',Result);
      E:=Pos('      </span>',Result);
   End;
End; //BlockClose

Function ParsePage_BingSearch(HTML:String):Cardinal; //BlockOpen

This part of the code is also needed, otherwise the script does not work.

I'm going to integrate the retreive of Plotkeywords in the base script of IMDB (although 100 seem to me many) and Connections when you tell me it is ready.
Now I do not have much time, but it is first in my ToDo list.

Recommendation:
Yes, Plotkeywords in the base script of IMDB and Connections as a secondary script.


As soon as the final version of the Connections script is finished. There are still a few tests to work properly for the final version.

Note: If Nostra wakes up and simply compiles the actual PVD with the libraries for HTTPS, we would have powerful scripts that could be quickly adapted to the oficial version.

It would be great if your wish in the future will also come true, let's wait and we'll see ...
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on December 15, 2018, 07:54:42 am
IMDB_ [EN] [HTTPS] script

Minor changes to the Title / OrigTitle code because there were minor changes in the source code of the page. The correction will be added to the next version of the IMDB_ [EN] [HTTPS] script.


Quote
Function ParsePage_IMDBMovieBASE(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit;
    //Retrieve: ~title~, ~year~, ~origtitle~, ~poster~ / ~imdbrating~, ~IMDB_Votes~ (Custom Field) / ~TOP_250~(Custom Field) /
    //          If Not(GET_FULL_CREDIT): ~crew~ctDirectors,ctWriters,ctComposers,ctProducers(Not in base page), ctActors
    //         ~description~ / ~category~ "keywords" / ~tagline~ / ~genre~
    //         If Not(GET_FULL_MPAA) ~mpaa~
    //         ~country~ / ~rdate~ in contry provider local IP geolocation
    //         If Not(GET_FULL_AKA) ~aka~.
    //         ~budget~ / ~money~ / ~studio~ "Production Co"
    //         If Not(GET_FULL_FEATURES) ~features~
  Var
      curPos,endPos,index:Integer;
      ItemValue,ItemList,ImageFile:String;
      titleValue:String;
      Name,Role,PersonURL:String;
      ItemArray:TWideArray;
  Begin
    LogMessage('Function ParsePage_IMDBMovieBASE BEGIN======================|');
    Result:=prFinished;  //It will change to prError if any big problem with exit;
    //Because the script don't retreives the data in the page order a token search for the first curPos position or block select is mandatory
    //Get ~title~, ~year~, ~origtitle~, ~poster~
    //Get all "raw" title summary (in raw because we need the hidden links, we avoid "complete" token in strings which opens/closes)
    ItemList:=TextBetWeenFirst(HTML,'<div class="title_block"','div class="plot_summary_wrapper">'); //WEB_SPECIFIC.
    If (Length(ItemList)>0) Then Begin    
      //titleValue:=TextBetWeenFirst(ItemList,'<div class="title_wrapper"><h1 class="">','<');       //Strings which opens/closes the data. WEB_SPECIFIC
      titleValue:=TextBetWeenFirst(ItemList,'<h1 class="">','<');       //Strings which opens/closes the data. WEB_SPECIFIC
      If titleValue = '0' then titleValue:='';
      If titleValue = '' then titleValue:=TextBetWeenFirst(ItemList,'<h1 class="long">','<');       //Strings which opens/closes the data. WEB_SPECIFIC

      AddFieldValueXML('title',titleValue);
      LogMessage('      Get result title:'+titleValue+'||');
        ItemValue:=TextBetWeenFirst(ItemList,'<span id="titleYear">(',')</span>            </h1>');                                 //Strings which opens/closes the data. WEB_SPECIFIC      
        AddFieldValueXML('year',ItemValue);
        LogMessage('      Get result year:'+ItemValue+'||');
        ItemValue:=TextBetWeenFirst(ItemList,'<div class="originalTitle">','<');          //Strings which opens/closes the data. WEB_SPECIFIC
        If (Length(ItemValue)=0) Then ItemValue:=titleValue;                           //Provider hides the original title if same that title. WEB_SPECIFIC
        AddFieldValueXML('origtitle',ItemValue);
        LogMessage('      Get result origtitle:'+ItemValue+'||');
        ItemValue:=TextBetWeenFirst(ItemList,BASE_URL_IMAGE_PRE_TRUE,'.');                 //Get poster code. Strings which opens/closes the data. WEB_SPECIFIC
        If (Length(ItemValue)>0) then begin
            ItemValue:=BASE_URL_IMAGE_PRE_TRUE + ItemValue;                             //Base poster URL without '.jpg'. WEB_SPECIFIC
            ImageFile:=GetAppPath+'Scripts\'+BASE_DOWNLOAD_FILE_IMAGE_NAME+'-Poster.jpg'
            // Avoid HTTPS redirection: Download https image to file  OutPutFile=
            If (1=DownloadImage(ItemValue + '._V1_UY' + IntToStr(MAX_IMAGE_HEIGTH) + '_.jpg',ImageFile)) then begin  //Dowload in the selected user max size. WEB_SPECIFIC
                //AddImageURL(itPoster,ImageFile);    //Get the photo from a file even if the next line we log the true URL.
                LogMessage('      Get result poster:'+ItemValue + '._V1_UY' + IntToStr(MAX_IMAGE_HEIGTH) + '_.jpg'+'||');
            end else if (1=DownloadImage(ItemValue +'.jpg',ImageFile)) then begin  //Dowload in the web base size. WEB_SPECIFIC
                //AddImageURL(itPoster,ImageFile);    //Get the photo from a file even if the next line we log the true URL.
                LogMessage('      Get result poster:' + ItemValue + '.jpg'+'||');
            end;       
        end;
    End;
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on December 22, 2018, 02:33:03 pm
IMDB_[EN][HTTPS]_(Connections) script

IMDB_[EN][HTTPS] (Connections) script is now renamed to IMDB_[EN][HTTPS]_(Connections) script and now this is the final version of the script.


IMDB_[EN][HTTPS] (Connections) script

I've edited the MOD version of IMDB_ [EN] [HTTPS] script, which is now called IMDB_ [EN] [HTTPS] (Connections) script to transfer the Movie Connections information to the Normal Connections PVD field.

An additional code has also been added to transfer the category, which is allowed to record
url titles in the Connections custom box for links to the Connections website. In the Connections custom field, your data record will look like this,
Quote
Followed by (2)   Version of (58)   Spin-off (1)   References (6)   Referenced in (27)   Features (1)   Featured in (1)   Spoofs (1)   Spoofed in (2)
it will also have a visible option in the record, clicking on it, in order to open the Internet window of this record in the web browser.

In case of only such a record,
Quote
Spin-off (1)
there will be no url link to the Connections website.

I've done this by fixing a link to a specific IMDb Movie Connections website in the Custom Connections field. Custom Connections field still has links to individual categories and how many records are in each category. Where there is only one category where no link has been made so far, it is now a direct link to a particular IMDb Movie Connections website.

New is now the Connect Custom Box, where you can now set the PVD view so that you can now see the categories and the number of movie titles and which movie addresses are in a particular category [for example, the following number of movie titles are followed: Follows (10) ].

Quote
  5. Connections field info names:   
     • Follows
     • Followed_by
     • Version_of
     • Remake_of
     • Remade_as
     • Edited_from
     • Edited_into
     • Spin_off_from (Spun_off_from)
     • Spin_off
     • References
     • Referenced_in
     • Features
     • Featured_in
     • Spoofs
     • Spoofed_in
These are all categories whose information can be transferred to the PVD database. Missing category " Alternate-language version of ", because I have noticed this category so far or found it with only one movie title.

Quote
  GET_FULL_CONNECTIONS  = True ;  //Download Connections provider page for retreive the MovieConnections info. Otherwise no info of the Connections provider page.
If you change this function, there is no transfer of any information.

Quote
  GET_FULL_CONNECT  = False ;  //Download Connections provider page for retreive the Follows, Followed_by, Version_of, Remake_of, Remade_as, Edited_from, Edited_into, Spin_off_from(Spun_off_from) and Spin_off info. Otherwise set to True to only the Follows and Followed by info of the Connections provider page.
If you change this function, the transfer of information will only be for the Follows and Followed by the info of the Connections provider page. You must also use the lower function you can not change, otherwise some other categories will be downloaded.

Quote
  GET_FULL_CONNECTION  = True ;  //Download Connections provider page for retreive the Follows, Followed_by, Version_of, Remake_of, Remade_as, Edited_from, Edited_into, Spin_off_from(Spun_off_from) and Spin_off info of the Connections provider page. Otherwise set to False to the full MovieConnections info.
If you change this feature, you will be transferring all the information on the Connections provider page.
Quote
Notice:
This function is basic set flagged-blocked with a " True " word.

Quote
Footnote:
You can change the functions according to your preferences, of course.

Quote
  CONNECT_LIMIT  = 100;    //Limit of number of connections retrieved.
The limit is basic set to the 100 records, because there may be several records in the same category as the limit is set. This number allows the script to run faster, although records may also be several times higher.

Also, filtering records with this part of the code is added, which allows the database to be visible only for movies and movies with the TV Movie tag in the normal OrigTitle PVD field.
Quote
If Pos('(Empty)',Title) OR Pos('(TV Episode)',OrigTitle) OR Pos('(TV Series)',OrigTitle) OR Pos('(TV Mini-Series)',OrigTitle) OR Pos('(Short)',OrigTitle) OR Pos('(TV Short)',OrigTitle) OR Pos('(TV Special)',OrigTitle) OR Pos('(Video Game)',OrigTitle) OR Pos('(Video)',OrigTitle) < 1 then
The following categories have such a filter:
Quote
     • Remake_of
     • Remade_as
     • Edited_into
     • Spin_off_from (Spun_off_from)
     • References
     • Referenced_in
     • Features
     • Featured_in
     • Spoofs
     • Spoofed_in

The following categories have in the database in addition to already visible records for movies and movies with the TV Movie tag as well as the TV Series tag in the normal OrigTitle PVD field.
Quote
     • Version_of
     • Edited_from

The following categories have in the database in addition to already visible records for movies and movies with the TV Movie tag as well as the TV Series and Video tag in the normal OrigTitle PVD field.
Quote
     • Follows
     • Followed_by
     • Spin_off

IMDB_ [EN] [HTTPS]_(Connections) script is attached.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on December 22, 2018, 03:30:39 pm
IMDB_ [EN] [HTTPS]_(Connections) script

Notice:


I added the OrigTitle code, which is essential for filtering records at all and does not change. In the Connection field, the notes will also have a year (for example, For Your Eyes Only (1981)), which is written only in the Original Title field, but not in the Titile field in the PVD database. This option is added so that you often see multiple identical titles of the title of a movie, but the years of the movie outcome are different. This is removed at that time (movie year) when you update the movie record and the Original Title checkbox is confirmed.

The complete code in IMDB_ [EN] [HTTPS] _ (Connections) script for downloading Connections information should not be added to the original IMDB_ [EN] [HTTPS] script, because it will greatly slow down the performance of the script. IMDB_ [EN] [HTTPS] _ (Connections) script should remain as user only an additional script that will be used only by those users who will, at their option, want to download Connections information in their PVD Movies database.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on January 03, 2019, 09:07:56 am
IMDB_[EN][HTTPS] (Connections) script

I'm going to integrate the retreive of Plotkeywords in the base script of IMDB (although 100 seem to me many) and Connections when you tell me it is ready.
Now I do not have much time, but it is first in my ToDo list.

Recommendation:
Yes, Plotkeywords in the base script of IMDB and Connections as a secondary script.

I mentioned this because, for example, The Spy Who Loved Me (https://www.imdb.com/title/tt0076752/) movie title had a problem and IMDB_ [EN] [HTTPS] (Connections) script did not find a Connections website, but it always reported that the page or url of the address could not be found.  The Connections web page url, however, was passed through a specific part of the code to the custom field, which proved that the page actually exists. Then I found a part of the code
Quote
Function DownloadPage(URL:AnsiString):String; //BlockOpen
//Returns the URL page text. If error returns empty string
  Var
    i:Integer;
    ScriptPath,WebText:String;
    Begin
.
.
.
    if (Pos('404 Error - IMDb',Result)>0) then begin  //Banned IP. WEB_SPECIFIC.
        If BYPASS_SILENT Then ShowMessage('The URL is not in use (404 Error - IMDb).'+Chr(13)+'Go to the provider web in order to in order to'+Chr(13)+'find the good page',SCRIPT_NAME);
        LogMessage(Chr(9)+Chr(9)+'      404 Error - IMDb|');
        Result:=''; //Fatal error
    End;
    //if (Pos('Page not found',Result)>0) then begin  //Banned IP. WEB_SPECIFIC.
    //    If BYPASS_SILENT Then ShowMessage('The URL is not in use (Page not found).'+Chr(13)+'Go to the provider web in order to in order to'+Chr(13)+'find the good page',SCRIPT_NAME);
    //    LogMessage(Chr(9)+Chr(9)+'      Page not found|');
    //    Result:=''; //Fatal error
    //End;
   
    if (Pos('405 Method not allowed',Result)>0) then begin  //Banned IP. WEB_SPECIFIC.
        If BYPASS_SILENT Then ShowMessage('The URL has HTTP method problems (405 Method not allowed).'+Chr(13)+'Go to the provider web in order to in order to'+Chr(13)+'find the good page',SCRIPT_NAME);      
        LogMessage(Chr(9)+Chr(9)+'      405 Method not allowed|');
        Result:=''; //Fatal error
    End;
.
.
.
    LogMessage(Chr(9)+Chr(9)+'Function DownloadPage END======================|');
    exit;
  End; //BlockClose
that prevented the detection of the website and I blocked it. The result of blocking this part of the code was that the IMDB_ [EN] [HTTPS] (Connections) script worked properly and correctly downloaded the data for this movie from the Connections website.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on January 03, 2019, 09:26:27 am
IMDB_ [EN] [HTTPS] script

Minor changes to the code.


Quote
Function ParsePage_IMDBMovieBASE(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit;
    //Retrieve: ~title~, ~year~, ~origtitle~, ~poster~ / ~imdbrating~, ~IMDB_Votes~ (Custom Field) / ~TOP_250~(Custom Field) /
    //          If Not(GET_FULL_CREDIT): ~crew~ctDirectors,ctWriters,ctComposers,ctProducers(Not in base page), ctActors
    //         ~description~ / ~category~ "keywords" / ~tagline~ / ~genre~
    //         If Not(GET_FULL_MPAA) ~mpaa~
    //         ~country~ / ~rdate~ in contry provider local IP geolocation
    //         If Not(GET_FULL_AKA) ~aka~.
    //         ~budget~ / ~money~ / ~studio~ "Production Co"
    //         If Not(GET_FULL_FEATURES) ~features~
  Var
      curPos,endPos,index:Integer;
      ItemValue1,ItemValue,ItemList,ImageFile:String;
      titleValue:String;
      Name,Role,PersonURL:String;
      ItemArray:TWideArray;
  Begin
    LogMessage('Function ParsePage_IMDBMovieBASE BEGIN======================|');
    Result:=prFinished;  //It will change to prError if any big problem with exit;
    //Because the script don't retreives the data in the page order a token search for the first curPos position or block select is mandatory   
    //Get ~title~, ~year~, ~origtitle~, ~poster~
    //Get all "raw" title summary (in raw because we need the hidden links, we avoid "complete" token in strings which opens/closes)
    ItemList:=TextBetWeenFirst(HTML,'<div class="title_block"','div class="plot_summary_wrapper">'); //WEB_SPECIFIC.
   If (Length(ItemList)>0) Then Begin    
   titleValue:=TextBetWeenFirst(ItemList,'<h1 class="">','<');       //Strings which opens/closes the data. WEB_SPECIFIC
      If titleValue = '0' then titleValue:='';
      If titleValue = '' then titleValue:=TextBetWeenFirst(ItemList,'<h1 class="long">','<');       //Strings which opens/closes the data. WEB_SPECIFIC
      AddFieldValueXML('title',titleValue);
      LogMessage('      Get result title:'+titleValue+'||'); 

        ItemValue:=TextBetWeenFirst(ItemList,'<span id="titleYear">(',')</span>            </h1>');                                 //Strings which opens/closes the data. WEB_SPECIFIC      
        AddFieldValueXML('year',ItemValue);
        LogMessage('      Get result year:'+ItemValue+'||');
        endPos:=PosFrom('title="See more release dates" >',ItemList,curPos);
      curPos:=PosFrom('title="See more release dates" >',ItemList,endPos);   
      endPos:=PosFrom('</a>            </div>',ItemList,curPos);   
      curPos:=curPos+43;
      ItemValue1:=Copy(ItemList,curPos,4);
      LogMessage('      Get result year00:'+ItemValue1+'||');
      ItemValue1:=StringReplace(ItemValue1,'/a> ','',True,False,True);
        if ItemValue1 <> '' then AddFieldValueXML('year',ItemValue1);
        if ItemValue1 <> '' then LogMessage('      Get result year0:'+ItemValue1+'||');   

        ItemValue:=TextBetWeenFirst(ItemList,'<div class="originalTitle">','<');          //Strings which opens/closes the data. WEB_SPECIFIC
        If (Length(ItemValue)=0) Then ItemValue:=titleValue;                           //Provider hides the original title if same that title. WEB_SPECIFIC
        AddFieldValueXML('origtitle',ItemValue);
        LogMessage('      Get result origtitle:'+ItemValue+'||');
        ItemValue:=TextBetWeenFirst(ItemList,BASE_URL_IMAGE_PRE_TRUE,'.');                 //Get poster code. Strings which opens/closes the data. WEB_SPECIFIC
        If (Length(ItemValue)>0) then begin
            ItemValue:=BASE_URL_IMAGE_PRE_TRUE + ItemValue;                             //Base poster URL without '.jpg'. WEB_SPECIFIC
            ImageFile:=GetAppPath+'Scripts\'+BASE_DOWNLOAD_FILE_IMAGE_NAME+'-Poster.jpg'
            // Avoid HTTPS redirection: Download https image to file  OutPutFile=
            If (1=DownloadImage(ItemValue + '._V1_UY' + IntToStr(MAX_IMAGE_HEIGTH) + '_.jpg',ImageFile)) then begin  //Dowload in the selected user max size. WEB_SPECIFIC
                //AddImageURL(itPoster,ImageFile);    //Get the photo from a file even if the next line we log the true URL.
                LogMessage('      Get result poster:'+ItemValue + '._V1_UY' + IntToStr(MAX_IMAGE_HEIGTH) + '_.jpg'+'||');
            end else if (1=DownloadImage(ItemValue +'.jpg',ImageFile)) then begin  //Dowload in the web base size. WEB_SPECIFIC
                //AddImageURL(itPoster,ImageFile);    //Get the photo from a file even if the next line we log the true URL.
                LogMessage('      Get result poster:' + ItemValue + '.jpg'+'||');
            end;       
        end;
    End;

Quote
    //Get ~script info~
    //Get ~rdate~ in contry provider local IP geolocation. See: http://sobizarre-en.blogspot.fr/2014/12/how-to-easily-defeat-imdb-geolocation.html
    //Get ~imdbrating~, ~IMDB_Votes~ (Two tries)
    ItemList:=TextBetWeenFirst(HTML,'<script type="application/ld+json"','}</script>'); //WEB_SPECIFIC.
    If (Length(ItemList)>0) Then Begin
      ItemValue:=TextBetWeenFirst(ItemList,'"@type": "','",');   //Strings which opens/closes the data. WEB_SPECIFIC      
      ItemValue:=StringReplace(ItemValue,'TVSeries','TV Series',True,False,True);
      ItemValue:=StringReplace(ItemValue,'TVEpisode','TV Episode',True,False,True);      
      //AddCustomFieldValueByName('IMDBMovietype',ItemValue);
      //LogMessage('      Get result MediaType (CF~IMDBMovietype~):'+ItemValue+'||');
      MediaType:=StringReplace(ItemValue,'TVSeries','TV Series',True,False,True);
      MediaType:=StringReplace(ItemValue,'TVEpisode','TV Episode',True,False,True);               
      AddCustomFieldValueByName('IMDB_Movietype',MediaType);
      LogMessage('      Get result MediaType (CF~IMDB_Movietype~):'+MediaType+'||');
      ItemValue:=TextBetWeenFirst(ItemList,'"contentRating": "','",');   //Strings which opens/closes the data. WEB_SPECIFIC
      AddCustomFieldValueByName('IMDB_MPAA',ItemValue);                                 
      if ItemValue <> '' then LogMessage('      Get result ContentRating (CF~IMDB_MPAA~):'+ItemValue+'||');
   ImageFile:=TextBetWeenFirst(ItemList,'"image": "','._V1_.jpg",')+'._V1_UY'+IntToStr(MAX_IMAGE_HEIGTH)+'_.jpg';
   AddImageURL(itPoster,ImageFile);
   LogMessage('      Get result poster:' + ImageFile +'||');
   ItemValue1:=TextBetWeenFirst(ItemList,'"datePublished": "','-');
   //AddFieldValueXML('year',ItemValue1);
    LogMessage('      Get result year01:'+ItemValue1+'||');

      ItemValue:=TextBetWeenFirst(ItemList,'"datePublished": "','",');   //Strings which opens/closes the data. WEB_SPECIFIC
        ItemValue:=StringReplace(ItemValue,'-01','-1',True,False,True);
        ItemValue:=StringReplace(ItemValue,'-02','-2',True,False,True);
        ItemValue:=StringReplace(ItemValue,'-03','-3',True,False,True);
        ItemValue:=StringReplace(ItemValue,'-04','-4',True,False,True);
        ItemValue:=StringReplace(ItemValue,'-05','-5',True,False,True);
        ItemValue:=StringReplace(ItemValue,'-06','-6',True,False,True);
        ItemValue:=StringReplace(ItemValue,'-07','-7',True,False,True);
        ItemValue:=StringReplace(ItemValue,'-08','-8',True,False,True);
        ItemValue:=StringReplace(ItemValue,'-09','-9',True,False,True);
      if ItemValue <> '' then LogMessage('      Get result ReleaseDatePublished: '+ItemValue+' ||');      
      if ItemValue <> '' then begin
            ExplodeString(ItemValue,ItemArray,'-');
            ItemValue:=ItemArray[2]+'.'+ ItemArray[1]+'.'+ItemArray[0];
         AddFieldValueXML('rdate',ItemValue);   
         AddCustomFieldValueByName('IMDB Release Date',ItemValue);   
            if ItemValue <> '' then LogMessage('      Get result DatePublished ~rdate~ (CF~IMDB Release Date~):'+ItemValue+'||');      
      End;
      ItemValue:=TextBetWeenFirst(ItemList,'"ratingCount": ',',');   //Strings which opens/closes the data. WEB_SPECIFIC
      AddCustomFieldValueByName('IMDB_Votes',ItemValue);      
      if ItemValue <> '' then LogMessage('      Get result ratingCount (~IMDB_Votes~): '+ItemValue+' ||');   
      ItemValue:=TextBetWeenFirst(ItemList,'"ratingValue": "','"');   //Strings which opens/closes the data. WEB_SPECIFIC
      AddFieldValueXML('imdbrating',ItemValue);
      AddCustomFieldValueByName('IMDB Rating',ItemValue);
      AddCustomFieldValueByName('IMDBRating',ItemValue);
      if ItemValue <> '' then LogMessage('      Get result ratingValue ~imdbrating~ (CF~IMDB Rating~~IMDBRating~): '+ItemValue+' ||');         
    End;
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on January 03, 2019, 09:39:59 am
IMDB_ [EN] [HTTPS] script

Minor changes to the code.


Quote
    //Get "plot_summary" as ~comment~ (In the PVD check fields table selection only ~description~ can be checked for overwrite but the user can change the config and
    // write ~description~ in ~comment~ so with the script parameter we Avoid double use of ~description~ and ~comment~.
    If Not(PRESERVE_PVD_DESCRIPTION_SETTING) Then Begin
        curPos:=Pos('<div class="plot_summary',HTML);                                 //WEB_SPECIFIC.   
        If 0<curPos then begin
            curPos:=Pos('<div class="summary_text">',HTML);                                 //WEB_SPECIFIC.
            If 0<curPos Then Begin
                //ItemValue:=TextBetWeen(HTML,'<div class="summary_text">','</div>',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
                ItemValue:=TextBetWeen(HTML,'<div class="summary_text','</div>',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
      curPos:=Pos('  <a href="',ItemValue);
               If 0<curPos then ItemValue:=Copy(ItemValue,0,curPos-1);

                ItemValue:=StringReplace(ItemValue,',            ',', ',True,False,True);
                ItemValue:=StringReplace(ItemValue,'            ',' ',True,False,True);
                ItemValue:=StringReplace(ItemValue,'        ',' ',True,False,True);
                ItemValue:=StringReplace(ItemValue,'Add a Plot »','',True,False,True);
                ItemValue:=StringReplace(ItemValue,'See full summary »','',True,False,True);
                ItemValue:=StringReplace(ItemValue,'">','',True,False,True);
                curPos:=Pos('var ue_t0=ue_t0||+new Date();var ',ItemValue);
                If 0<curPos then ItemValue:=Copy(ItemValue,0,curPos-1);
                AddFieldValueXML('comment',ItemValue);
                LogMessage('      Get result "Plot_summary" as ~comment~:'+ItemValue+'||');
            End;
        End;   
    End;


New part of the added code.
Quote
//USER FUNCTIONS AND PROCEDURES==================================================================================BlockClose
//OBLIGATORY CALLBACK PRINCIPAL FUNCTION=========================================================================BlockOpen
Function ParsePage(HTML:String;URL:AnsiString):Cardinal; //BlockOpen
  Var
    MovieID,titleValue,yearValue:String;
    ResultTmp:Cardinal;
    Date:String;
   Fullinfo,Movie_URL,IMDB_URL: String;
    DateParts: TWideArray;
  Begin
.
.
.
//Parse with the Movie URL 'smNormal'------------------------------------------------------------------------------------
    If (Mode=smNormal) Then Begin
        //Get the script Overwrite Options saved in pvdconf.ini (Remember that PVD only save the options on exit)
        //0=Do nothing,1=Set if Empty,2=Overwrite
        PVDConfigOptions:=TextBetWeenFirst(FileToString(GetAppPath+'pvdconf.ini'),'IMDB_[EN][HTTPS].psf=',Chr(13));
            //LogMessage('Stored Script PVDConfigOptions is:'+PVDConfigOptions+'||');
            //LogMessage('The Script option of position '+ IntToStr(opPoster) +' is:'+Copy(PVDConfigOptions,opPoster,1)+'||');
        Result:=prFinished;  //It will change to prError if any big problem with exit; 
        //Information in several page. Needs Provider MovieID
        MovieID:=TextBetWeenFirst(DownloadURL,BASE_URL_PRE_TRUE,BASE_URL_SUF);
        LogMessage('  ParsePage mode smNormal|'+IntToStr(Mode)+'|. Getting provider data for MovieID|'+ MovieID +'|');
        AddCustomFieldValueByName('IMDbID',MovieID);
        LogMessage('      Get result IMDbID (CF~IMDbID~):'+MovieID+'||');
        //Get ~url~
        if (0=Pos(BASE_URL_PRE,StoredURL)) then begin   //Write the url if not exists
            AddFieldValueXML('url',StringReplace(DownloadURL,BASE_URL_PRE_TRUE,BASE_URL_PRE,True,False,False));
            LogMessage('      Get result url:'+StringReplace(DownloadURL,BASE_URL_PRE_TRUE,BASE_URL_PRE,True,False,False)+'||');
        end;
           //Get ~IMDb Movie Url~       
        Movie_URL:=StringReplace(DownloadURL,BASE_URL_PRE_TRUE,BASE_URL_PRE,True,False,False);
        LogMessage('      Get result Movie_url: '+Movie_URL+' ||');
   Fullinfo:='';
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'">Maindetails</link>  ';
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'reference">Reference</link>  ';
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'fullcredits">Full Cast&Crew</link>  ';
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'awards">Awards</link>  ';
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'releaseinfo">ReleaseDates</link>  ';
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'technical">TechnicalSpecs</link>  ';
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'parentalguide">ParentsGuide</link>  ';   
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'keywords">PlotKeywords</link>  ';
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'plotsummary">PlotSummary</link>  ';
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'locations">FilmLocations</link>  ';   
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'taglines">Taglines</link>  ';
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'companycredits">CompanyCredits</link>  ';
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'movieconnections">Connections</link>  ';
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'soundtrack">Soundtracks</link>  ';
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'externalsites">Miscellaneous</link>  ';
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'mediaindex">PhotoGallery</link>';
      If Fullinfo <> '' then IMDB_URL:=Fullinfo;
      If IMDB_URL <> '' then AddCustomFieldValueByName('IMDb Movie Url',IMDB_URL);
      LogMessage('      Get result Fullinfo-IMDB_URL (CF~IMDb Movie Url~ ):'+#13+IMDB_URL+'||');    
   
    //Parse principal provider page = BASE_URL_PRE_TRUE------------------------------------------------------------------
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on January 05, 2019, 03:10:50 pm
Attached temporal Acumulative Scripts Patch date: 2019.01.05 (to copy in PVD folder for overwriting the existing files).

Changelog:
        IMDB_[EN][HTTPS]
                  V 1.4.0.0 (05/01/2019) Ivek23: Added ParsePage_PLOTKEYWORDS. Ivek23 corrections since last version (see forum).


Notes for Ivek23:
Tell me if I was wrong in something.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: ronaldomm on January 05, 2019, 11:31:41 pm
Hi,

I have been a long time user of this excellent program, but I am with a problem about the filmography of actors in this new version.
After I used the IMDB People [EN] [HTTPS] plugin for an specific actor, it isn't showing his full filmography. Look the exemple attached.

Please, could you help me in this case?

Regards.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on January 06, 2019, 12:36:34 pm
Hello,
Thanks for your alert, you are right: nowadays the "IMDB People [EN] [HTTPS]" don't retrieve the full actor filmography from IMDB and only shows the fimography of the movies stored in the PVD.
There are several reasons for it:
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on January 06, 2019, 12:58:33 pm
Hello,
Thanks for your alert, you are right: nowadays the "IMDB People [EN] [HTTPS]" don't retrieve the full actor filmography from IMDB and only shows the fimography of the movies stored in the PVD.
There are several reasons for it:
  • The full person filmography can be in five pages. For instance F. Murray Abraham can be: Actor | Soundtrack | Thanks | Self | Archive footage
  • When you add filmography, PVD create a record for the movie even if it isn't in your collection. For instance, F. Murray Abraham has 188 credits!
  • Finaly, the lack of time for this feature not very much demanded. But it is a good opportunity for you to try to make this script. We will help you with it!

I agree with VVV_Easy. I have been waiting for some time for someone to ask a question about the transfer of filmography information to the IMDb people script. I have also been thinking of inserting code into IMDb people script for downloading filmography in the database. I'll see what to help.

Temporary help if you want to have a filmography can also be an IMDb People plugin that still works with Proxomitron  (http://www.videodb.info/forum_en/index.php/topic,4083.0.html)if you know it or have already used it. This is a temporary solution until it is ready code for filmography transfer.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: ronaldomm on January 06, 2019, 03:36:00 pm
 :)
OK. Thanks for while.
I am going to wait by the definitive solution. For while, i keep using The Proxomitron.
See you.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on January 06, 2019, 08:07:37 pm
Hi,

I have been a long time user of this excellent program, but I am with a problem about the filmography of actors in this new version.
After I used the IMDB People [EN] [HTTPS] plugin for an specific actor, it isn't showing his full filmography. Look the exemple attached.

Please, could you help me in this case?

Regards.
:)
OK. Thanks for while.
I am going to wait by the definitive solution. For while, i keep using The Proxomitron.
See you.

The solution is already, but it still needs some tests before the release, which you can expect in a day or two.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on January 07, 2019, 11:21:30 am
Tell me if I was wrong in something.

I have not yet been able to verify everything if something is missing or it may not be necessary to change anything at the moment.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on January 07, 2019, 09:09:27 pm
The solution is already, but it still needs some tests before the release, which you can expect in a day or two.

IMDB_People_ [EN] [HTTPS]+ script now has an added filmography code for transmitting information as well as for Altenative names.

IMDB_People_ [EN] [HTTPS]+ script is attached.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on January 08, 2019, 09:50:19 am
IMDB_People_ [EN] [HTTPS]+ script

Quote
Function ParsePage_IMDBPeopleAWARDS(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit
    //Retrieve: AddAward(Event, Award, Category, Recipient, Year, Won)
  Var
    curPos,endPos,endPosAux,index,curPos0,curPos1,curPos2,curPos3,curPos4,endPos0,endPos1,endPos2:Integer;
    MovieURL,ItemList:String;
    ItemArray: TWideArray;
    EventBlock,EventName,EventYear,YearBlock,AwardBlock,AwardName,AwardCategory,AwardRecipient,MovieYear:String;
    AwardWon: Boolean;
  Begin
    LogMessage('Function ParsePage_IMDBPeopleAWARDS BEGIN=====================||');
    Result:=prFinished;  //It will change to prError if any big problem with exit   
.
.
.
                //Recipient Level     
                curPos2:=Pos('<td class="award_description">',AwardBlock);                       //String which opens the AwardCategory and AwardRecipient list data. WEB_SPECIFIC
                While curPos2>0 Do Begin
                    //Get all "raw" list data for create good values separators (not use TextBetWeen)
                    endPos2:=PosFrom('</td>',AwardBlock,curPos2)+Length('</td>');                 //Strings which opens/closes the data. WEB_SPECIFIC
                    ItemList:=Copy(AwardBlock,curPos2,endPos2-curPos2);
                    //LogMessage('           Parse results ('+IntToStr(curPos2)+','+IntToStr(curPos2)+') complex ItemList:'+ItemList+'||');
                    //The Receipt awards ItemList may have:  1) empty description or not have name (not interesting) and break ItemArray[]. 2) Several titles with year 3) Detail o full Notes
                    //So is better search sequentily by token in a block than with ItemArray
                    endPosAux:=PosFrom(#13,ItemList,2);                                        //Strings which opens/closes the data. WEB_SPECIFIC
                    curPos3:=PosFrom('title',ItemList,2);                                       //Strings which opens/closes the data. WEB_SPECIFIC
                    If (endPosAux<curPos3) Or (curPos3=0) Then Begin                            //There is Awardcategory because #13 is befor name or there isn't name. WEB_SPECIFIC
                        curPos4:=1;
                        AwardCategory:=TextBetWeen(ItemList,'<td class="award_description">',#13,false,curPos4);   //Strings which opens/closes the data. WEB_SPECIFIC
                        LogMessage('     Parse Results in AwardCategory:'+AwardCategory+'||');
                        curPos4:=Pos('Shared with:',AwardCategory);                          //WEB_SPECIFIC.
                        If 0<curPos4 then AwardCategory:=Copy(AwardCategory,0,curPos4-1);
                        LogMessage('     Parse Results in AwardCategory0:'+AwardCategory+'||');

                    End Else Begin
                        AwardCategory:='';
                    End;
                    If curPos3=0 Then Begin //Award without Recipient
                    AddAward(EventName, AwardName, AwardCategory, '', EventYear, AwardWon);
                        LogMessage('      Get results Awards:#'+IntToStr(index)+'|'+EventName+'|'+AwardName+'|'+AwardCategory+'|'+''+'|'+EventYear+'|'); //+BoolToStr(AwardWon)+'||');
                    End;
                    While curPos3>0 Do Begin   
                        MovieURL:='http://www.imdb.com/title'+TextBetWeen(ItemList,'<a href="/title','?ref_=nmawd_awd_',true,curPos4)+'/';                                      //Strings which opens/closes the data. WEB_SPECIFIC
                        LogMessage('  **  Parse Results in MovieURL: '+MovieURL);

                        AwardRecipient:=TextBetWeen(ItemList,'>','<',false,curPos3);              //Strings which opens/closes the data. WEB_SPECIFIC
                  LogMessage('      Parse Results in AwardRecipient:'+AwardRecipient+'||');
                        MovieYear:=TextBetWeen(ItemList,'(',')',false,curPos3);                  //Strings which opens/closes the data. WEB_SPECIFIC
                        LogMessage('  **  Parse Results in MovieYear:'+MovieYear);

                        AddAward(EventName, AwardName, AwardCategory, AwardRecipient, EventYear, AwardWon);
                        LogMessage('      Get results Awards:#'+IntToStr(index)+'|'+EventName+'|'+AwardName+'|'+AwardCategory+'|'+AwardRecipient+'|'+EventYear+'|'); //+BoolToStr(AwardWon)+'||');                   
                        //endPosAux:=PosFrom('note',ItemList,curPos3);                            //Strings which opens/closes the data. WEB_SPECIFIC
                  endPosAux:=PosFrom('truncated-note',ItemList,curPos3);                            //Strings which opens/closes the data. WEB_SPECIFIC
                        curPos3:=PosFrom('title',ItemList,curPos3);                              //Strings which opens/closes the data. WEB_SPECIFIC

                  //curPos3:=PosFrom('name',ItemList,curPos3);                              //Strings which opens/closes the data. WEB_SPECIFIC
                        If curPos3>endPosAux Then curPos3:=0                                   //Avoid Names in notes. WEB_SPECIFIC                                                                                                                                     
                    End;
                    curPos2:=PosFrom('<td class="award_description">',AwardBlock,endPos2);        //String which opens the AwardCategory and AwardRecipient list data. WEB_SPECIFIC
                End;
                curPos1:=PosFrom('<td class="award_outcome"',YearBlock,endPos1);               //String which opens the AwardName and Won list data. WEB_SPECIFIC
            End;
            curPos0:=PosFrom('<td class="award_year"',EventBlock,endPos0);                      //String which opens the AwardYearlist data. WEB_SPECIFIC
        End;
        curPos:=PosFrom('<table class="awards"',HTML,endPos);                               //String which detectecs the Event. Search directly '<h3>' is very inconsistent. WEB_SPECIFIC
        index:=index+1;
    End;
    LogMessage('Function ParsePage_IMDBMovieAWARDS END=====================||');
  End; //BlockClose
In ParsePage_IMDBPeopleAWARDS, a few months ago, beauty errors were discovered or found, which had just been corrected or removed. IMDB_People_ [EN] [HTTPS]+ script now correctly transfers the Awards informations.

The revised IMDB_People_ [EN] [HTTPS]+ script  is attached.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on January 19, 2019, 09:03:46 pm
Tell me if I was wrong in something.

I have not yet been able to verify everything if something is missing or it may not be necessary to change anything at the moment.

In the near future, I will post corrections in the existing IMDB_ [EN] [HTTPS] script, which will be called IMDB_ [EN] [HTTPS] + script (in a few days, hopefully it will be),  as well as some new scripts such as Plot Summary, Soundtracks and other information. It will also be published separately IMDB_ [EN] [HTTPS] script for Cast & Crew extra informations.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on January 21, 2019, 03:19:51 pm
Notes for Ivek23:
  • Not included the parameter GET_PLOTKEYWORDS_LIMIT=False because is redundant with PLOTKEYWORDS_LIMIT set in a big number
  • Not included the script part "Connections" as suggested (but we can include it with parameter GET_CONNECTIONS=False)
That's OK.

Notes for Ivek23:
  • Corrected: "curPos:=curPos+43;" by "curPos:=curPos+45;" in order to get the good year (at least in Amadeus movie)
  • Not included the code part of Post (http://www.videodb.info/forum_en/index.php/topic,4134.msg20865.html#msg20865):
I will explain this below.

Notes for Ivek23:
  • Not included the code part of Post (http://www.videodb.info/forum_en/index.php/topic,4134.msg20865.html#msg20865):

       ImageFile:=TextBetWeenFirst(ItemList,'"image": "','._V1_.jpg",')+'._V1_UY'+IntToStr(MAX_IMAGE_HEIGTH)+'_.jpg';
       AddImageURL(itPoster,ImageFile);
       LogMessage('      Get result poster:' + ImageFile +'||');
       ItemValue1:=TextBetWeenFirst(ItemList,'"datePublished": "','-');
       //AddFieldValueXML('year',ItemValue1);
        LogMessage('      Get result year01:'+ItemValue1+'||');


    because is not needed for download the poster and the AddFieldValueXML is after // (commentary) and ItemValue1 is not more used
It can be added there, it works. I added it for updating another IMDb script, where I used it and works the same way as here. But it does not need to be updated every time, when the next time they will encounter new changes to the source code for the IMDb website.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on January 21, 2019, 03:30:30 pm
IMDB_ [EN] [HTTPS]

New added:

Quote
  5. Cast & Crew field info names:   
   •   Directed by
   •   Writing Credits  // NO (WGA)
   •   Cast (in credits order)  // Only data info in normal Actors field
   •   Produced by
   •   Music by
   •   Cinematography by  // NO
   •   Film Editing by  // NO
   •   Casting By  // NO
   •   Production Design by  // NO
   •   Art Direction by  // NO
   •   Set Decoration by  // NO
   •   Costume Design by  // NO
   •   Production Management  // NO
   •   Second Unit Director or Assistant Director  // NO 
   •   Special Effects by  // NO
   •   Visual Effects by  // NO
   •   Casting Department  // NO
   •   Location Management  // NO
   •   Music Department  // NO
   •   Thanks
Quote
  GET_CREW_CREDIT  = False ;  //Download Cast and Credit provider page for retreive the some credits (crew) info. Otherwise set to False to only the Directors, Writers, Composers, Producers, Cinematography and Thanks info of the Cast and Credit provider page. 
  //GET_CREW_CREDIT  = True ;  //Download Cast and Credit provider page for retreive the some credits (crew) info. Otherwise set to False to only the Directors, Writers, Composers, Producers, Cinematography and Thanks info of the Cast and Credit provider page.
Works in IMDB_ [EN] [HTTPS] _ (Cast & Crew) script.
Quote
  GET_FULL_PLOTSUMMARY  = True ;  //Download Plot provider page for retreive the Summaries info. Otherwise only the info of the principal movie page

Modified:

Function ParsePage_IMDBMovieBASE
Quote
Function ParsePage_IMDBMovieBASE(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit;
    //Retrieve: ~title~, ~year~, ~origtitle~, ~poster~ / ~imdbrating~, ~IMDB_Votes~ (Custom Field) / ~TOP_250~(Custom Field) /
    //          If Not(GET_FULL_CREDIT): ~crew~ctDirectors,ctWriters,ctComposers,ctProducers(Not in base page), ctActors
    //         ~description~ / ~category~ "keywords" / ~tagline~ / ~genre~
    //         If Not(GET_FULL_MPAA) ~mpaa~
    //         ~country~ / ~rdate~ in contry provider local IP geolocation
    //         If Not(GET_FULL_AKA) ~aka~.
    //         ~budget~ / ~money~ / ~studio~ "Production Co"
    //         If Not(GET_FULL_FEATURES) ~features~
  Var
      curPos,endPos,index:Integer;
      ItemValue1,ItemValue0,ItemValue,ItemList,ImageFile:String;
      titleValue:String;
      Name,Role,PersonURL:String;
      ItemArray:TWideArray;
  Begin
    LogMessage('Function ParsePage_IMDBMovieBASE BEGIN======================|');
    Result:=prFinished;  //It will change to prError if any big problem with exit;  //**// (* *)
    //Because the script don't retreives the data in the page order a token search for the first curPos position or block select is mandatory
    //Get ~title~, ~year~, ~origtitle~, ~poster~
    //Get all "raw" title summary (in raw because we need the hidden links, we avoid "complete" token in strings which opens/closes)
    ItemList:=TextBetWeenFirst(HTML,'<div class="title_block"','div class="plot_summary_wrapper">'); //WEB_SPECIFIC.
   //LogMessage('      Get result ItemList:'+#13+ItemList+'||');   
    If (Length(ItemList)>0) Then Begin
        //Get ~title~
        titleValue:=TextBetWeenFirst(ItemList,'<h1 class="">','<');       //Strings which opens/closes the data. WEB_SPECIFIC
        If titleValue = '0' then titleValue:='';
        If titleValue = '' then titleValue:=TextBetWeenFirst(ItemList,'<h1 class="long">','<');       //Strings which opens/closes the data. WEB_SPECIFIC
        AddFieldValueXML('title',titleValue);
        LogMessage('      Get result title:'+titleValue+'||');   
        //First normal kind get of ~year~
        ItemValue:=TextBetWeenFirst(ItemList,'<span id="titleYear">(',')</span>            </h1>');                                 //Strings which opens/closes the data. WEB_SPECIFIC      
        AddFieldValueXML('year',ItemValue);
        LogMessage('      Get result normal year:'+ItemValue+'||');
      //Second kind of get of ~year~
      endPos:=PosFrom('title="See more release dates" >',ItemList,curPos);
        curPos:=PosFrom('title="See more release dates" >',ItemList,endPos);
      endPos:=PosFrom(' (',ItemList,curPos);
      if (EndPos < 1) OR (EndPos > PosFrom('</a>            </div>',ItemList,curPos)) then
      EndPos := PosFrom('</a>            </div>',ItemList,curPos);
      curPos:=endPos-4;
      ItemValue0:=Copy(ItemList,curPos,4);
      LogMessage('      Get result second kind of year 0:'+ItemValue0+'||');
        if ItemValue0 <> '' then AddFieldValueXML('year',ItemValue0);
        if ItemValue0 <> '' then LogMessage('      Get result second kind of year:'+ItemValue0+'||');      
        //Second kind of get of ~year~ or for series
        endPos:=PosFrom('title="See more release dates" >',ItemList,curPos);
        curPos:=PosFrom('title="See more release dates" >',ItemList,endPos);   
        endPos:=PosFrom('</a>            </div>',ItemList,curPos);
      curPos:=curPos+43;
        ItemValue1:=Copy(ItemList,curPos,4);
        LogMessage('      Get series result for series year:'+ItemValue1+'||');
        //ItemValue1:=StringReplace(ItemValue1,'/a> ','',True,False,True);
        if ItemValue1 <> '' then AddFieldValueXML('year',ItemValue1);
        if ItemValue1 <> '' then LogMessage('      Get series result for series year2:'+ItemValue1+'||');

        //Get ~origtitle~
        ItemValue:=TextBetWeenFirst(ItemList,'<div class="originalTitle">','<');          //Strings which opens/closes the data. WEB_SPECIFIC
        If (Length(ItemValue)=0) Then ItemValue:=titleValue;                           //Provider hides the original title if same that title. WEB_SPECIFIC
        AddFieldValueXML('origtitle',ItemValue);
        LogMessage('      Get result origtitle:'+ItemValue+'||');
        ItemValue:=TextBetWeenFirst(ItemList,BASE_URL_IMAGE_PRE_TRUE,'.');                 //Get poster code. Strings which opens/closes the data. WEB_SPECIFIC
        If ((Length(ItemValue)>0) and Not(USE_SAVED_PVDCONFIG and (Copy(PVDConfigOptions,opPoster,1)='0'))) then begin  //The Poster will be saved in PVDthen begin
            ItemValue:=BASE_URL_IMAGE_PRE_TRUE + ItemValue;                             //Base poster URL without '.jpg'. WEB_SPECIFIC
            ImageFile:=GetAppPath+'Scripts\'+BASE_DOWNLOAD_FILE_IMAGE_NAME+'-Poster.jpg'
            // Avoid HTTPS redirection: Download https image to file  OutPutFile=
            If (1=DownloadImage(ItemValue + '._V1_UY' + IntToStr(MAX_IMAGE_HEIGTH) + '_.jpg',ImageFile)) then begin  //Dowload in the selected user max size. WEB_SPECIFIC
                AddImageURL(itPoster,ImageFile);    //Get the photo from a file even if the next line we log the true URL.
                LogMessage('      Get result poster:'+ItemValue + '._V1_UY' + IntToStr(MAX_IMAGE_HEIGTH) + '_.jpg'+'||');
            end else if (1=DownloadImage(ItemValue +'.jpg',ImageFile)) then begin  //Dowload in the web base size. WEB_SPECIFIC
                AddImageURL(itPoster,ImageFile);    //Get the photo from a file even if the next line we log the true URL.
                LogMessage('      Get result poster:' + ItemValue + '.jpg'+'||');
            end;
        End;
    End;

Now I've updated the year code. For users who want to update information for series, adds the year correctly to the database.

Quote
    //Get ~script info~
    //Get ~rdate~ in contry provider local IP geolocation. See: http://sobizarre-en.blogspot.fr/2014/12/how-to-easily-defeat-imdb-geolocation.html
    //Get ~imdbrating~, ~IMDB_Votes~ (Two tries)
    ItemList:=TextBetWeenFirst(HTML,'<script type="application/ld+json"','}</script>'); //WEB_SPECIFIC.
    If (Length(ItemList)>0) Then Begin
      ItemValue:=TextBetWeenFirst(ItemList,'"@type": "','",');   //Strings which opens/closes the data. WEB_SPECIFIC      
      ItemValue:=StringReplace(ItemValue,'TVSeries','TV Series',True,False,True);
      ItemValue:=StringReplace(ItemValue,'TVEpisode','TV Episode',True,False,True);      
      //AddCustomFieldValueByName('IMDBMovietype',ItemValue);
      //LogMessage('      Get result MediaType (CF~IMDBMovietype~):'+ItemValue+'||');
      MediaType:=StringReplace(ItemValue,'TVSeries','TV Series',True,False,True);
      MediaType:=StringReplace(ItemValue,'TVEpisode','TV Episode',True,False,True);               
      AddCustomFieldValueByName('IMDB_Movietype',MediaType);
      LogMessage('      Get result MediaType (CF~IMDB_Movietype~):'+MediaType+'||');
      ItemValue:=TextBetWeenFirst(ItemList,'"contentRating": "','",');   //Strings which opens/closes the data. WEB_SPECIFIC
      AddCustomFieldValueByName('IMDB_MPAA',ItemValue);                                 
      if ItemValue <> '' then LogMessage('      Get result ContentRating (CF~IMDB_MPAA~):'+ItemValue+'||');
   ImageFile:=TextBetWeenFirst(ItemList,'"image": "','._V1_.jpg",')+'._V1_UY'+IntToStr(MAX_IMAGE_HEIGTH)+'_.jpg';
   AddImageURL(itPoster,ImageFile);
   LogMessage('      Get result poster:' + ImageFile +'||');
   ItemValue1:=TextBetWeenFirst(ItemList,'"datePublished": "','-');
   //AddFieldValueXML('year',ItemValue1);
    LogMessage('      Get result year01:'+ItemValue1+'||');

      ItemValue:=TextBetWeenFirst(ItemList,'"datePublished": "','",');   //Strings which opens/closes the data. WEB_SPECIFIC
        ItemValue:=StringReplace(ItemValue,'-01','-1',True,False,True);
        ItemValue:=StringReplace(ItemValue,'-02','-2',True,False,True);
        ItemValue:=StringReplace(ItemValue,'-03','-3',True,False,True);
        ItemValue:=StringReplace(ItemValue,'-04','-4',True,False,True);
        ItemValue:=StringReplace(ItemValue,'-05','-5',True,False,True);
        ItemValue:=StringReplace(ItemValue,'-06','-6',True,False,True);
        ItemValue:=StringReplace(ItemValue,'-07','-7',True,False,True);
        ItemValue:=StringReplace(ItemValue,'-08','-8',True,False,True);
        ItemValue:=StringReplace(ItemValue,'-09','-9',True,False,True);
      if ItemValue <> '' then LogMessage('      Get result ReleaseDatePublished: '+ItemValue+' ||');      
      if ItemValue <> '' then begin
            ExplodeString(ItemValue,ItemArray,'-');
            ItemValue:=ItemArray[2]+'.'+ ItemArray[1]+'.'+ItemArray[0];
         AddFieldValueXML('rdate',ItemValue);   
         AddCustomFieldValueByName('IMDB Release Date',ItemValue);   
            if ItemValue <> '' then LogMessage('      Get result DatePublished ~rdate~ (CF~IMDB Release Date~):'+ItemValue+'||');      
      End;
      ItemValue:=TextBetWeenFirst(ItemList,'"ratingCount": ',',');   //Strings which opens/closes the data. WEB_SPECIFIC
      AddCustomFieldValueByName('IMDB_Votes',ItemValue);      
      if ItemValue <> '' then LogMessage('      Get result ratingCount (~IMDB_Votes~): '+ItemValue+' ||');   
      ItemValue:=TextBetWeenFirst(ItemList,'"ratingValue": "','"');   //Strings which opens/closes the data. WEB_SPECIFIC
      AddFieldValueXML('imdbrating',ItemValue);
      AddCustomFieldValueByName('IMDB Rating',ItemValue);
      AddCustomFieldValueByName('IMDBRating',ItemValue);
      if ItemValue <> '' then LogMessage('      Get result ratingValue ~imdbrating~ (CF~IMDB Rating~~IMDBRating~): '+ItemValue+' ||');         
    End;

Notes for Ivek23:
  • Not included the code part of Post (http://www.videodb.info/forum_en/index.php/topic,4134.msg20865.html#msg20865):

       ImageFile:=TextBetWeenFirst(ItemList,'"image": "','._V1_.jpg",')+'._V1_UY'+IntToStr(MAX_IMAGE_HEIGTH)+'_.jpg';
       AddImageURL(itPoster,ImageFile);
       LogMessage('      Get result poster:' + ImageFile +'||');
       ItemValue1:=TextBetWeenFirst(ItemList,'"datePublished": "','-');
       //AddFieldValueXML('year',ItemValue1);
        LogMessage('      Get result year01:'+ItemValue1+'||');


    because is not needed for download the poster and the AddFieldValueXML is after // (commentary) and ItemValue1 is not more used
It can be added there, it works. I added it for updating another IMDb script, where I used it and works the same way as here. But it does not need to be updated every time, when the next time they will encounter new changes to the source code for the IMDb website.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on January 21, 2019, 04:12:37 pm
IMDB_ [EN] [HTTPS] script

Modified:


Quote
    //Get "plot_summary" as ~comment~ (In the PVD check fields table selection only ~description~ can be checked for overwrite but the user can change the config and
    // write ~description~ in ~comment~ so with the script parameter we Avoid double use of ~description~ and ~comment~.
    If Not(PRESERVE_PVD_DESCRIPTION_SETTING) Then Begin
        curPos:=Pos('<div class="plot_summary',HTML);                                 //WEB_SPECIFIC.   
        If 0<curPos then begin
            curPos:=Pos('<div class="summary_text">',HTML);                                 //WEB_SPECIFIC.
            If 0<curPos Then Begin
                ItemValue:=TextBetWeen(HTML,'<div class="summary_text','</div>',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
                curPos:=Pos('  <a href="',ItemValue);
                If 0<curPos then ItemValue:=Copy(ItemValue,0,curPos-1);
                ItemValue:=StringReplace(ItemValue,',            ',', ',True,False,True);
                ItemValue:=StringReplace(ItemValue,'            ',' ',True,False,True);
                ItemValue:=StringReplace(ItemValue,'        ',' ',True,False,True);
                ItemValue:=StringReplace(ItemValue,'Add a Plot »','',True,False,True);
                ItemValue:=StringReplace(ItemValue,'See full summary »','',True,False,True);
                ItemValue:=StringReplace(ItemValue,'">','',True,False,True);
                curPos:=Pos('var ue_t0=ue_t0||+new Date();var ',ItemValue);
                If 0<curPos then ItemValue:=Copy(ItemValue,0,curPos-1);
                //AddFieldValueXML('comment',ItemValue);
                LogMessage('      Get result "Plot_summary" as ~comment~:'+ItemValue+'||');
            End;
        End;   
    End;
Now the plot_summary info on the primary IMDb page will no longer be written to the comment field, although you will make changes to the PVD settings.

Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on January 21, 2019, 04:13:59 pm
IMDB_ [EN] [HTTPS] script

ParsePage_IMDBMovieCREDIT

New added and modified:


Quote
Function ParsePage_IMDBMovieCREDIT(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit;
    //Retrieve: ~crew~ctDirectors,ctWriters,ctComposers,ctProducers
    //          ~actors~ ctActors
  Var
    curPos,endPos,index:Integer;
   debug_pos1:Integer;
    ItemValue,ItemList:String;
    Name,Role,PersonURL:String;
   ItemValue14:String;
   Fullinfo,Directors,Writers,Composers,Producers:String;   

  Begin
    LogMessage('Function ParsePage_IMDBMovieCREDIT BEGIN=====================||');
    Result:=prFinished;  //It will change to prError if any big problem with exit;
  //Get Full Cast & Crew   
   curPos:=Pos('<h1 class="header">Full Cast & Crew</h1>',HTML);
   If 0<curPos Then Begin   
      EndPos:=curPos;
      ItemValue:=HTMLValues2(HTML,'<h1 class="header">Full Cast & Crew</h1>','<h2>See also</h2>','<h4 ','</h4>','<br>   •   ',EndPos);  //WEB_SPECIFIC.
      If ItemValue <> '' then ItemValue:='<link url="'+GetFieldValueXML('url')+'fullcredits">Full Cast&Crew info names</link><br>   •   '+ItemValue;
      AddCustomFieldValueByName('Cinema1',ItemValue);
      LogMessage('      Get result Movie Full Cast & Crew Credits:'+ItemValue+'||');
   End;      
   
  //Get ~crew~ctDirectors,ctWriters,ctComposers,ctProducers
   Fullinfo:='';
    //Go to "Directed by" ~crew~ctDirectors
    curPos:=Pos('<h4 class="dataHeaderWithBorder">Directed by',HTML);                     //Strings start which opens the block content data. WEB_SPECIFIC   
    If 0<curPos Then Begin
      curPos:=PosFrom('</h4>',HTML,curPos);                                              //Strings end which opens the block content data.  WEB_SPECIFIC
      curPos:=curPos+Length('</h4>');                                                    //Strings end which opens the block content data.  WEB_SPECIFIC
      //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~ n the ~crew~
      endPos:=PosFrom('</table>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
      ItemList:=Copy(HTML,curPos,endPos-curPos);
      //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');
      curPos:=Pos('<td class="name">',ItemList)                                         //String which opens the subList data. WEB_SPECIFIC
      index:=1;
      While curPos>0 Do Begin
         If (index>PEOPLE_LIMIT) Then break;     //Limited depassed.
         //Get PersonURL (Always must it has)
         PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF;     //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
         //Get Name (Always must it has)
         Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC     
         LogMessage('      Parse Results Name:'+Name+'||');
         //Get Role: PVD don't save Role in crew people but is added for Ivek23 as LogMessage compatibility
            Role:=TextBetWeen(ItemList,'<td ','</td>',false,curPos);                             //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results Role1:'+Role+'||');
         Role:=StringReplace(Role,'class="credit">','',True,False,True);
         Role:=StringReplace(Role,'colspan="2">','',True,False,True);
         Role:=StringReplace(Role,'            (uncredited)','(uncredited)',True,False,True);
         LogMessage('      Parse Results Role:'+Role+'||');

         AddMoviePerson(Name,'','',PersonURL,ctDirectors);
         //LogMessage('      Get results Director:#'+IntToStr(index)+'|'+Name+'|'+PersonURL+'||ctDirectors');
         if Directors <> '' then Directors:=Directors+#13;
         if PersonURL <> '' then Directors:=Directors+'<link url="'+PersonURL+'">'+Name+'</link>'+'          ';
         if Role <> '' then Directors:=Directors+Role;         
         LogMessage('      Get results Directors:#'+IntToStr(index)+'|'+Name+'|'+Role+'|'+PersonURL+'||ctDirectors');

         curPos:=PosFrom('<td class="name">',ItemList,curPos)                                            //String which opens the subList data. WEB_SPECIFIC
         index:=index+1;
      End;
    End;
   If Directors <> '' then fullinfo:=fullinfo+'Directed:'+'<br>'+Directors+'<br><br>';
    //Go to "Writer:" or "Writers:" ~crew~ctWriters
   curPos:=Pos('<h4 class="dataHeaderWithBorder">Writing Credits',HTML);
   If 0<curPos Then Begin   
      curPos:=PosFrom('</h4>',HTML,curPos);                                              //Strings end which opens the block content data.  WEB_SPECIFIC
      curPos:=curPos+Length('</h4>');                                                    //Strings end which opens the block content data.  WEB_SPECIFIC
      //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~
      endPos:=PosFrom('</table>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
      ItemList:=Copy(HTML,curPos,endPos-curPos);
      //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');
      curPos:=Pos('<td class="name">',ItemList)                                            //String which opens the subList data. WEB_SPECIFIC
      index:=1;
      While curPos>0 Do Begin
         If (index>PEOPLE_LIMIT) Then break;     //Limited depassed.
         //Get PersonURL (Always must it has)
         PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF;     //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
         //Get Name (Always must it has)
         Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results Name:'+Name+'||');
            //Get Role: PVD don't save Role in crew people but is added for Ivek23 as LogMessage compatibility
         Role:=TextBetWeen(ItemList,'<td class="credit">','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC
         Role := StringReplace(Role,') and',') ',True,False,True);
         Role := StringReplace(Role,') &',') ',True,False,True);
         Role := StringReplace(Role,'novel &','novel ',True,False,True);
         Role := StringReplace(Role,'screenplay &','screenplay ',True,False,True);
         Role := StringReplace(Role,'script &','script ',True,False,True);
         //Role := StringReplace(Role,'story &','story ',True,False,True);
         //Role := StringReplace(Role,'&',' ',True,False,True);
         if Pos('and',Role) = 1 then Delete(Role, 1, 3);
         if Pos('&',Role) = 1 then Delete(Role, 1, 2);
         if Pos('story and',Role) = 1 then Delete(Role, 7, 3);
         //Role:=StringReplace(Role,'            ','          ',True,False,True);         
         LogMessage('      Parse Results Role:'+Role+'||');      
      
         AddMoviePerson(Name,'','',PersonURL,ctWriters);
         //LogMessage('      Get results Writer:#'+IntToStr(index)+'|'+Name+'|'+PersonURL+'||ctWriters');
         if Writers <> '' then Writers:=Writers+#13;
         if PersonURL <> '' then Writers:=Writers+'<link url="'+PersonURL+'">'+Name+'</link>'+'          ';
         if Role <> '' then Writers:=Writers+Role;                  
         LogMessage('      Get results Writers:#'+IntToStr(index)+'|'+Name+'|'+Role+'|'+PersonURL+'||ctWriters'); 
         curPos:=PosFrom('<td class="name">',ItemList,curPos)     //String which opens the subList data. WEB_SPECIFIC
         index:=index+1;
      End;
    End;
   If Writers <> '' then fullinfo:=fullinfo +'Writing Credits:'+'<br>'+Writers+'<br><br>';
    //Go to "Producers" ~crew~ctProducers
   curPos:=Pos('<h4 class="dataHeaderWithBorder">Produced by',HTML);
   If 0<curPos Then Begin
      curPos:=PosFrom('</h4>',HTML,curPos);                                              //Strings end which opens the block content data.  WEB_SPECIFIC
      curPos:=curPos+Length('</h4>');                                                    //Strings end which opens the block content data.  WEB_SPECIFIC
      //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~
      endPos:=PosFrom('</table>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
      ItemList:=Copy(HTML,curPos,endPos-curPos);
      //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');;
      curPos:=Pos('<td class="name">',ItemList)                                               //String which opens the subList data. WEB_SPECIFIC
      index:=1;
      While curPos>0 Do Begin
         If (index>PEOPLE_LIMIT) Then break;     //Limited depassed.
         //Get PersonURL (Always must it has)
         PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF;     //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
         //Get Name (Always must it has)
         Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results Name:'+Name+'||');
            //Get Role: PVD don't save Role in crew people but is added for Ivek23 as LogMessage compatibility
         Role:=TextBetWeen(ItemList,'<td class="credit">','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results Role:'+Role+'||');

         AddMoviePerson(Name,'','',PersonURL,ctProducers);
         //LogMessage('      Get results Producers:#'+IntToStr(index)+'|'+Name+'|'+PersonURL+'||ctProducers');
         if Producers <> '' then Producers:=Producers+#13;
         if PersonURL <> '' then Producers:=Producers+'<link url="'+PersonURL+'">'+Name+'</link>'+'          ';
         if Role <> '' then Producers:=Producers+Role;         
         LogMessage('      Get results Producers:#'+IntToStr(index)+'|'+Name+'|'+Role+'|'+PersonURL+'||ctProducers'); 

         curPos:=PosFrom('<td class="name">',ItemList,curPos)                                                   //String which opens the subList data. WEB_SPECIFIC
         index:=index+1;
      End;   
    End;
   If Producers <> '' then fullinfo:=fullinfo +'Produced:'+'<br>'+Producers+'<br><br>';
    //Go to "Music" ~crew~ctComposers
   curPos:=Pos('<h4 class="dataHeaderWithBorder">Music by',HTML);
   If 0<curPos Then Begin
      curPos:=PosFrom('</h4>',HTML,curPos);                                              //Strings end which opens the block content data.  WEB_SPECIFIC
      curPos:=curPos+Length('</h4>');                                                    //Strings end which opens the block content data.  WEB_SPECIFIC
      //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~
      endPos:=PosFrom('</table>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
      ItemList:=Copy(HTML,curPos,endPos-curPos);
      //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');List+'||');
      curPos:=Pos('<td class="name">',ItemList)                                               //String which opens the subList data. WEB_SPECIFIC
      index:=1;
      While curPos>0 Do Begin
         If (index>PEOPLE_LIMIT) Then break;     //Limited depassed.
         //Get PersonURL (Always must it has)
         PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF;     //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
         //Get Name (Always must it has)
         Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC         
         LogMessage('      Parse Results Name:'+Name+'||');
            //Get Role: PVD don't save Role in crew people but is added for Ivek23 as LogMessage compatibility
         Role:=TextBetWeen(ItemList,'<td class="credit">','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results Role:'+Role+'||');
         AddMoviePerson(Name,'',Role,PersonURL,ctComposers);
         //LogMessage('      Get results Composers:#'+IntToStr(index)+'|'+Name+'|'+PersonURL+'||ctComposers');
         if Composers <> '' then Composers:=Composers+#13;
         if PersonURL <> '' then Composers:=Composers+'<link url="'+PersonURL+'">'+Name+'</link>'+'          ';
         if Role <> '' then Composers:=Composers+Role;
         LogMessage('      Get results Composers:#'+IntToStr(index)+'|'+Name+'|'+Role+'|'+PersonURL+'||ctComposers'); 

         curPos:=PosFrom('<td class="name">',ItemList,curPos)                                                   //String which opens the subList data. WEB_SPECIFIC
         index:=index+1;
      End;
    End;
   If Composers <> '' then fullinfo:=fullinfo +'Music:'+'<br>'+Composers+'<br><br>';
    //Get  to "Cast" ~actors~ ctActors
    //Go Cast list
    curPos:=Pos('<table class="cast_list">',HTML);                                     //Strings start which opens the block content data. WEB_SPECIFIC
    curPos:=curPos+Length('<table class="cast_list">');                                //Strings end which opens the block content data.  WEB_SPECIFIC
    //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~
    endPos:=PosFrom('</div>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
    ItemList:=Copy(HTML,curPos,endPos-curPos);
    //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');
    curPos:=Pos('<td class="primary_photo">',ItemList)                               //String which opens the subList data. WEB_SPECIFIC
    index:=1;
    While curPos>0 Do Begin
        If (index>PEOPLE_LIMIT) Then break;     //Limited depassed.
        //Get PersonURL (Always must it has)
        PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF; //Strings which opens/closes the data. WEB_SPECIFIC
        LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
        //Get Name (Always must it has)
        Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                             //Strings which opens/closes the data. WEB_SPECIFIC      
        LogMessage('      Parse Results Name:'+Name+'||');
        //Get Role
        Role:=TextBetWeen(ItemList,'<td class="character">','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC         
        LogMessage('      Parse Results Role:'+Role+'||');
      Role:=StringReplace(Role,'       (',' (',True,False,True);
      Role:=StringReplace(Role,'(uncredited)                    ','(uncredited) • ',True,False,True);
      Role:=StringReplace(Role,')                    ',') ',True,False,True);
      Role:=StringReplace(Role,' /              ',' - ',True,False,True);
      Role:=StringReplace(Role,'         / ...                    ',' • ',True,False,True);
      Role:=StringReplace(Role,'                  ',' • ',True,False,True);      
      Role:=StringReplace(Role,'      (uncredited)',' (uncredited)',True,False,True);
      LogMessage('      Parse Results Role_1:'+Role+'||');      
        AddMoviePerson(Name,'',Role,PersonURL,ctActors);
        LogMessage('      Get results cast:#'+IntToStr(index)+'|'+Name+'|'+Role+'|'+PersonURL+'||ctActors');                                               
        curPos:=PosFrom('<td class="primary_photo">',ItemList,curPos);                                  //String which opens the Web Result item List data. WEB_SPECIFIC
        index:=index+1;
    End;   
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on January 21, 2019, 04:15:52 pm
IMDB_ [EN] [HTTPS] script

ParsePage_IMDBMovieCREDIT

Part 2:

New added and modified:


Quote
Function ParsePage_IMDBMovieCREDIT(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit;
    //Retrieve: ~crew~ctDirectors,ctWriters,ctComposers,ctProducers
    //          ~actors~ ctActors
  Var
    curPos,endPos,index:Integer;
   debug_pos1:Integer;
    ItemValue,ItemList:String;
    Name,Role,PersonURL:String;
   ItemValue14:String;
   Fullinfo,Directors,Writers,Composers,Producers:String;   

  Begin
    LogMessage('Function ParsePage_IMDBMovieCREDIT BEGIN=====================||');
    Result:=prFinished;  //It will change to prError if any big problem with exit;
.
.
.
   //Get to "Others Crew"
    //Go Others Crew list
   //~Cinematography~
   ItemValue:='';
   curPos:=Pos('<h4 class="dataHeaderWithBorder">Cinematography by',HTML);
   If 0<curPos Then Begin
      curPos:=PosFrom('</h4>',HTML,curPos);
      curPos:=curPos+Length('</h4>');
      endPos:=PosFrom('</table>',HTML,curPos);
      ItemList:=Copy(HTML,curPos,endPos-curPos);
      //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');
      curPos:=Pos('<td class="name">',ItemList);
      index:=1;
      While curPos>0 Do Begin
         PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF;
         //LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
         Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);
         //LogMessage('      Parse Results Name:'+Name+'||');
         Role:=TextBetWeen(ItemList,'<td class="credit">','</td>',false,curPos);
         //LogMessage('      Parse Results Role:'+Role+'||');   
         if ItemValue <> '' then ItemValue:=ItemValue+#13;
         if PersonURL <> '' then ItemValue:=ItemValue+'<link url="'+PersonURL+'">'+Name+'</link>'+'          ';
         if Role <> '' then ItemValue:=ItemValue+Role;
         //AddCustomFieldValueByName('Cinema',ItemValue);   
         //if ItemValue <> '' then Fullinfo:=Fullinfo+'Cinematography:<br>'+ItemValue+'<br><br>';
         LogMessage('      Get results Cinematography by:#'+IntToStr(index)+'|'+Name+'|'+Role+'|'+PersonURL+'||');                                               
         curPos:=PosFrom('<td class="name">',ItemList,curPos);
         index:=index+1;
      End;   
   End;
   If ItemValue <> '' then Fullinfo:=Fullinfo+'Cinematography:<br>'+ItemValue+'<br><br>';       
   //~Thanks~
   ItemValue14:='';
   curPos:=Pos('<h4 class="dataHeaderWithBorder">Thanks',HTML);
   If 0<curPos Then Begin
      curPos:=PosFrom('</h4>',HTML,curPos);
      curPos:=curPos+Length('</h4>');
      endPos:=PosFrom('</table>',HTML,curPos);
      ItemList:=Copy(HTML,curPos,endPos-curPos);
      //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');
      curPos:=Pos('<td class="name">',ItemList);
      index:=1;
      While curPos>0 Do Begin
         PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF;               
         //LogMessage('      Parse Results PersonURL:'+PersonURL+'||');
         Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);
         //LogMessage('      Parse Results Name:'+Name+'||');
         Role:=TextBetWeen(ItemList,'<td class="credit">','</td>',false,curPos);
         //LogMessage('      Parse Results Role:'+Role+'||');   
         //if ItemValue14 <> '' then ItemValue14:=ItemValue14+#13;
         if ItemValue14 <> '' then ItemValue14:=ItemValue14+#32#32;   
         If Pos('the producers wish to thank',Role) OR Pos('special thanks',Role) OR Pos('thanks',Role) < 1 then   
         if PersonURL <> '' then ItemValue14:=ItemValue14+'<link url="'+PersonURL+'">'+Name+'</link>'+'          ';            
         debug_pos1:=Pos('the producers wish to thank',Role);
            If debug_pos1 >0 then Role:=Copy(Role,0,debug_pos1-1);
         debug_pos1:=Pos('very special thanks',Role);
            If debug_pos1 >0 then Role:=Copy(Role,0,debug_pos1-1);            
         debug_pos1:=Pos('special thanks',Role);
            If debug_pos1 >0 then Role:=Copy(Role,0,debug_pos1-1);
         debug_pos1:=Pos('thanks',Role);
            If debug_pos1 >0 then Role:=Copy(Role,0,debug_pos1-1);            
         if Role <> '' then ItemValue14:=ItemValue14+Role;
         ItemValue14:=StringReplace(ItemValue14,'      <link url="',#13+'<link url="',True,False,True);
         LogMessage('      Get results Thanks:#'+IntToStr(index)+'|'+Name+'|'+Role+'|'+PersonURL+'||');            
         curPos:=PosFrom('<td class="name">',ItemList,curPos);    
         index:=index+1;
      End;   
   End;   
   If ItemValue14 <> '' then Fullinfo:=Fullinfo+'Thanks:<br>'+ItemValue14;      
   if (Length(Fullinfo)>0) then begin
        AddCustomFieldValueByName('Cinema',Fullinfo);
        LogMessage('      Get result Movie Others Crew:'+Fullinfo+'||');
    End;

    LogMessage('Function ParsePage_IMDBMovieCREDIT END=====================||');
  End; //BlockClose
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on January 21, 2019, 04:34:30 pm
IMDB_ [EN] [HTTPS]  script

New added:


Quote
Function ParsePage_IMDBMoviePLOTSUMMARY(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit
    //Retrieve: PLOTSUMMARY~
  Var
    //curPos:Integer;
    curPos,endPos:Integer;
   ItemValue,ItemList:String;
   Category,Name:String;
  Begin
    LogMessage('Function ParsePage_IMDBMoviePLOTSUMMARY BEGIN=====================||');
    Result:=prFinished;  //It will change to prError if any big problem with exit;
   // ~Plot~ info
    curPos:=Pos('<h1 class="header">Plot</h1>',HTML);
    if (curPos=0) then Exit;
    EndPos:=curPos;   
   //Get "Plot" info
   //If Pos('<h1 class="header">Plot</h1>',HTML)>0 Then Begin
   curPos:=Pos('<h1 class="header">Plot</h1>',HTML);
   If 0<curPos Then Begin   
      EndPos:=curPos;   
      Category:=TextBetween(HTML,'<h1 class="header">','</h1>',True,curPos);
      LogMessage('      Get result Category:'+Category+'||');
      Name:=TextBetween(HTML,'<div class="ipl-itemcount-header">','</ul>',True,curPos);
      LogMessage('      Get result Name:'+Name+'||');
      Name:=StringReplace(Name,'                Jump to:         ',#13#160#160#160#160#9679#160#160,True,False,True);
      //if Name <> '' then Name:=#160#160#160#9679#160#160#160+Name;
      if Name <> '' then Name:=#160#160#160#160+Name;
      LogMessage('      Get result Name1:'+Name+'||');
      AddCustomFieldValueByName('Plot Summary','<link url="'+GetFieldValueXML('url')+'plotsummary">'+Category+' Summary</link>'+Name);      
   End;      
   // ~Plot Summary~
   If (Pos('It looks like we don'+Chr(39)+'t have any Summaries for this title yet.',HTML)>0) then Exit;   
   //Get "Summaries" info (~Comment~)
   If Pos('<h1 class="header">Plot</h1>',HTML)>0 Then Begin
      curPos:=Pos('<h1 class="header">Plot</h1>',HTML);
      EndPos:=curPos;   
      ItemValue:=HTMLValues2(HTML,'Plot','See also','<li class="ipl-zebra-list__item" id="summary','</p>','<br><br>-------<br><br>',EndPos);   
      ItemValue:=StringReplace(ItemValue,'<a href='+Chr(39),'<link url="'+GetFieldValueXML('url'),True,False,True);
      ItemValue:=StringReplace(ItemValue,'?ref_='+Chr(39)+'>',+'">',True,False,True);
      ItemValue:=StringReplace(ItemValue,'</a>','</link>',True,False,True);         
      AddFieldValueXML('comment',ItemValue);
      LogMessage('      Get result ItemValue (~Comment~):'+ItemValue+'||');   
   End;   
   // ~Plot~ info ~Plot Summary~
    curPos:=Pos('<div id="pagecontent" class="pagecontent">',HTML);
    curPos:=PosFrom('<section class="article listo">',HTML,curPos);
    curPos:=curPos+Length('<div id="content-2-wide" class="redesign">');
    endPos:=PosFrom('</section>',HTML,curPos);
    ItemList:=Copy(HTML,curPos,endPos-curPos);   
   //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');         
    LogMessage('Function ParsePage_IMDBMoviePLOTSUMMARY END=====================||');
  End; //BlockClose   

Quote
//USER FUNCTIONS AND PROCEDURES==================================================================================BlockClose
//OBLIGATORY CALLBACK PRINCIPAL FUNCTION=========================================================================BlockOpen
Function ParsePage(HTML:String;URL:AnsiString):Cardinal; //BlockOpen
  Var
    MovieID,titleValue,yearValue:String;
    ResultTmp:Cardinal;
    Date:String;
    Fullinfo,Movie_URL,IMDB_URL: String;
    DateParts: TWideArray;
  Begin
    LogMessage('Function ParsePage BEGIN======================|');
    LogMessage('Global Var-Mode|'+IntToStr(Mode)+'|');
    LogMessage('Global Var-DownloadURL|'+DownloadURL+'|');
    LogMessage('      Local Var-URL|'+URL+'|');
.
.
.
    //Parse Plot Keywords provider page = BASE_URL_PLOTKEYWORDS---------------------------------------------------------------------------
        If GET_FULL_PLOTKEYWORDS Then Begin
            DownloadURL:=StringReplace(BASE_URL_PLOTKEYWORDS,'%IMDB_ID',MovieID,True,True,False);
            HTML:=DownloadPage(DownloadURL);  //True page for parsing
            HTML:=HTMLToText(HTML);
            ResultTmp:=ParsePage_IMDBMoviePLOTKEYWORDS(HTML);
         //AddCustomFieldValueByName('Download_URL','<link url="'+DownloadURL+'">IMDB PLOTKEYWORDS</link>  ');
            If Not(ResultTmp=prFinished) then Result:=ResultTmp;
        End;      
    //Parse Plotsummary provider page =  BASE_URL_PLOTSUMMARY---------------------------------------------------------------------------
        If GET_FULL_PLOTSUMMARY Then Begin
            DownloadURL:=StringReplace(BASE_URL_PLOTSUMMARY,'%IMDB_ID',MovieID,True,True,False);
            HTML:=DownloadPage(DownloadURL);  //True page for parsing
            HTML:=HTMLToText(HTML);
            ResultTmp:=ParsePage_IMDBMoviePLOTSUMMARY(HTML);
         AddCustomFieldValueByName('Plotsummary_URL','<link url="'+DownloadURL+'">IMDB_PLOTSUMMARY</link>');
            If Not(ResultTmp=prFinished) then Result:=ResultTmp;
        End;         
    //Date ~Updated~ (choose simple or verbose version)
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on January 21, 2019, 04:41:04 pm
IMDB_ [EN] [HTTPS] script

Modified:


Quote
//USER FUNCTIONS AND PROCEDURES==================================================================================BlockClose
//OBLIGATORY CALLBACK PRINCIPAL FUNCTION=========================================================================BlockOpen
Function ParsePage(HTML:String;URL:AnsiString):Cardinal; //BlockOpen
  Var
    MovieID,titleValue,yearValue:String;
    ResultTmp:Cardinal;
    Date:String;
   Fullinfo,Movie_URL,IMDB_URL: String;
    DateParts: TWideArray;
  Begin
.
.
.
//Parse with the Movie URL 'smNormal'------------------------------------------------------------------------------------
    If (Mode=smNormal) Then Begin
        //Get the script Overwrite Options saved in pvdconf.ini (Remember that PVD only save the options on exit)
        //0=Do nothing,1=Set if Empty,2=Overwrite
        PVDConfigOptions:=TextBetWeenFirst(FileToString(GetAppPath+'pvdconf.ini'),'IMDB_[EN][HTTPS].psf=',Chr(13));
            //LogMessage('Stored Script PVDConfigOptions is:'+PVDConfigOptions+'||');
            //LogMessage('The Script option of position '+ IntToStr(opPoster) +' is:'+Copy(PVDConfigOptions,opPoster,1)+'||');
        Result:=prFinished;  //It will change to prError if any big problem with exit; 
        //Information in several page. Needs Provider MovieID
        MovieID:=TextBetWeenFirst(DownloadURL,BASE_URL_PRE_TRUE,BASE_URL_SUF);
        LogMessage('  ParsePage mode smNormal|'+IntToStr(Mode)+'|. Getting provider data for MovieID|'+ MovieID +'|');
        AddCustomFieldValueByName('IMDbID',MovieID);
        LogMessage('      Get result IMDbID (CF~IMDbID~):'+MovieID+'||');
        //Get ~url~
        if (0=Pos(BASE_URL_PRE,StoredURL)) then begin   //Write the url if not exists
            AddFieldValueXML('url',StringReplace(DownloadURL,BASE_URL_PRE_TRUE,BASE_URL_PRE,True,False,False));
            LogMessage('      Get result url:'+StringReplace(DownloadURL,BASE_URL_PRE_TRUE,BASE_URL_PRE,True,False,False)+'||');
        end;
        //Get ~IMDb Movie Url~       
        Movie_URL:=StringReplace(DownloadURL,BASE_URL_PRE_TRUE,BASE_URL_PRE,True,False,False);
        LogMessage('      Get result Movie_url: '+Movie_URL+' ||');
        Fullinfo:='';
      //Fullinfo:=Fullinfo+'<link url="'+Movie_UR+'">Maindetails</link>  ';
        Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'">MainPage</link>  ';
        Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'reference">Reference</link>  ';
        Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'fullcredits">Full Cast&Crew</link>  ';
        Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'awards">Awards</link>  ';
        Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'releaseinfo">ReleaseDates</link>  ';
        //Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'technical">TechnicalSpecs</link>  ';
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'technical">TechSpecs</link>  ';
        Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'parentalguide">ParentsGuide</link>  ';   
        Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'keywords">PlotKeywords</link>  ';
        Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'plotsummary">PlotSummary</link>  ';
        Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'locations">FilmLocations</link>  ';   
        Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'taglines">Taglines</link>  ';
        Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'companycredits">CompanyCredits</link>  ';
        Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'movieconnections">Connections</link>  ';
        Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'soundtrack">Soundtracks</link>  ';
        //Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'externalsites">Miscellaneous</link>  ';
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'externalsites">MiscSites</link>  ';
      Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'externalsites#photos">MiscPhoto</link>  ';
        Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'mediaindex">PhotoGallery</link>  ';
      //Fullinfo:=Fullinfo+'<link url="'+Movie_URLL+'externalsites#misc">MiscSites</link>  ';
      //Fullinfo:=Fullinfo+'<link url="'+Movie_URL+'externalsites#photos">MiscPhoto</link>  ';
        If Fullinfo <> '' then IMDB_URL:=Fullinfo;
        If IMDB_URL <> '' then AddCustomFieldValueByName('IMDb Movie Url',IMDB_URL);
        LogMessage('      Get result Fullinfo-IMDB_URL (CF~IMDb Movie Url~ ):'+#13+IMDB_URL+'||');
   
    //Parse principal provider page = BASE_URL_PRE_TRUE------------------------------------------------------------------

All these changes are included in IMDB_ [EN] [HTTPS] + script, which is attached.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on January 21, 2019, 05:00:47 pm
IMDB_[EN][HTTPS] (Plotsummary) script

An IMDB_ [EN] [HTTPS] (Plotsummary) script for full download of Plotsummary information was made.

Plotsummary information is transferred to the comment field.

IMDB_ [EN] [HTTPS] (Plotsummary) script is attached.


You can also find more information about Custom Fields in the IMDB_ [EN] [HTTPS] (Plotsummary) script Possibles improving Custom Fields working in MOD version. (http://www.videodb.info/forum_en/index.php/topic,4141.msg20789.html#msg20789)
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on January 21, 2019, 06:40:03 pm
IMDB_[EN][HTTPS]_(Cast&Crew) script

Something has already been described for Cast & Crew downloading information on this link.
http://www.videodb.info/forum_en/index.php/topic,4134.msg20881.html#msg20881
http://www.videodb.info/forum_en/index.php/topic,4134.msg20883.html#msg20883

and
http://www.videodb.info/forum_en/index.php/topic,4134.msg20884.html#msg20884

IMDB_ [EN] [HTTPS]+ script has the default downloading the following information.

   •   Directed by
   •   Writing Credits  // NO (WGA)
   •   Cast (in credits order)  // Only data info in normal Actors field
   •   Produced by
   •   Music by
   •   Cinematography by
   •   Thanks


Here are some more details, which are only possible with IMDB_ [EN] [HTTPS] _ (Cast & Crew) script.


Quote
  5. Cast & Crew field info names:   
   •   Directed by
   •   Writing Credits  // NO (WGA)
   •   Cast (in credits order)  // Only data info in normal Actors field
   •   Produced by
   •   Music by
   •   Cinematography by
   •   Film Editing by
   •   Casting By
   •   Production Design by
   •   Art Direction by
   •   Set Decoration by
   •   Costume Design by
   •   Production Management
   •   Second Unit Director or Assistant Director 
   •   Special Effects by
   •   Visual Effects by
   •   Casting Department
   •   Location Management
   •   Music Department
   •   Thanks

IMDB_[EN][HTTPS]_(Cast&Crew) script has the default settings for downloading the following information.

Default settings
Quote
  //GET_CREW_CREDIT  = False ;  //Download Cast and Credit provider page for retreive the some credits (crew) info. Otherwise set to False to only the Directors, Writers, Composers, Producers, Cinematography and Thanks info of the Cast and Credit provider page. 
  GET_CREW_CREDIT  = True ;  //Download Cast and Credit provider page for retreive the some credits (crew) info. Otherwise set to False to only the Directors, Writers, Composers, Producers, Cinematography and Thanks info of the Cast and Credit provider page.
  GET_FULL_CREW_CREDIT  = False ;  //Download Cast and Credit provider page for retreive the more  some credits (crew) info. Otherwise set to False to only the Directors, Writers, Composers, Producers and Cinematography + (GET_CREW_CREDIT = True) info of the Cast and Credit provider page.

IMDB_ [EN] [HTTPS]+ script downloading the following information.
   •   Directed by
   •   Writing Credits  // NO (WGA)
   •   Cast (in credits order)  // Only data info in normal Actors field
   •   Produced by
   •   Music by
   •   Cinematography by
   •   Thanks


IMDB_ [EN] [HTTPS] _ (Cast & Crew) script also has the default downloading the following information.

   •   Film Editing by
   •   Casting By
   •   Production Design by
   •   Art Direction by
   •   Set Decoration by
   •   Costume Design by
   •   Production Management
   •   Second Unit Director or Assistant Director


IMDB_ [EN] [HTTPS] _ (Cast & Crew) script has default blocked settings for downloading the following information.

   •   Special Effects by
   •   Visual Effects by
   •   Casting Department
   •   Location Management
   •   Music Department


To have unblocked settings for downloading the following information, change the blocked settings in IMDB_ [EN] [HTTPS] _ (Cast & Crew) script as shown below.

Default settings
Quote
  //GET_CREW_CREDIT  = False ;  //Download Cast and Credit provider page for retreive the some credits (crew) info. Otherwise set to False to only the Directors, Writers, Composers, Producers, Cinematography and Thanks info of the Cast and Credit provider page. 
  GET_CREW_CREDIT  = True ;  //Download Cast and Credit provider page for retreive the some credits (crew) info. Otherwise set to False to only the Directors, Writers, Composers, Producers, Cinematography and Thanks info of the Cast and Credit provider page.
  GET_FULL_CREW_CREDIT  = True ;  //Download Cast and Credit provider page for retreive the more  some credits (crew) info. Otherwise set to False to only the Directors, Writers, Composers, Producers and Cinematography + (GET_CREW_CREDIT = True) info of the Cast and Credit provider page.

Then, IMDB_ [EN] [HTTPS] _ (Cast & Crew) script will download all of the possible information described.

   •   Directed by
   •   Writing Credits  // NO (WGA)
   •   Cast (in credits order)  // Only data info in normal Actors field
   •   Produced by
   •   Music by
   •   Cinematography by
   •   Film Editing by
   •   Casting By
   •   Production Design by
   •   Art Direction by
   •   Set Decoration by
   •   Costume Design by
   •   Production Management
   •   Second Unit Director or Assistant Director 
   •   Special Effects by
   •   Visual Effects by
   •   Casting Department
   •   Location Management
   •   Music Department
   •   Thanks


Custom fields are Cinema and Cinema1.

]IMDB_[EN][HTTPS]_(Cast&Crew) script is attached.


You can also find more information about Custom Fields in the IMDB_ [EN] [HTTPS] _ (Cast & Crew) script in Possibles improving Custom Fields working in MOD version. (http://www.videodb.info/forum_en/index.php/topic,4141.msg20789.html#msg20789)

Notification:

IMDB_ [EN] [HTTPS] _ (Cast & Crew) script should only be a secondary script for those users whose interest in this information is.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on February 10, 2019, 11:25:10 am
Attached temporal Acumulative Scripts Patch date: 2019.02.10 (to copy in PVD folder for overwriting the existing files).

Changelog:
        IMDB_[EN][HTTPS]
                  V 1.4.1.0 (10/02/2019) Ivek23: Added ParsePage_PLOTSUMMARY.Ivek23 corrections and improvements since last version (see forum)
        IMDB_People_[EN][HTTPS]
                 V 1.3.1.0 (10/02/2019) Ivek23: Ivek23 corrections and improvements since last version (see forum)

Notes for Ivek23:
I have tried to include all the fixes and improvements in this patch except the alternative scripts, for example IMDB_[EN][HTTPS]_(Cast&Crew) script.
I think that you are better positioned to decide the evolution of the script because I only use TheMovieDb and FilmAffinity, then in the future, please, change the versions of the main IMDB scripts (IMDB_[EN][HTTPS] and IMDB_People_[EN][HTTPS]) and I will include its in the MOD version same as you publish it.
The alternative scripts can be left in the forum only for interested people.

Tell me if you are agreee and if something is wrong in the patch. Thank you for your hard and good work.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on February 10, 2019, 06:35:24 pm
IMDB_People_[EN][HTTPS]

Why is not an added code for alternative names for people?
Why is not an added code for filmography for people?

If the code for both of them is added, IMDB_People_ [EN] [HTTPS] script will download all the information and will be the right and complete replacement for the IMDB People plugin.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on February 10, 2019, 06:59:29 pm
IMDB_ [EN] [HTTPS] script

The alternative scripts can be left in the forum only for interested people.

I agree with this proposal.

Alternatives IMDB_ [EN] [HTTPS] scripts for interested people will be available in the future in a new topic, where they will be all descriptions for them. In this topic, however, any errors and descriptions of possible corrections for them will still be reported.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on February 14, 2019, 08:22:33 am
IMDB_ [EN] [HTTPS] script

Tell me if you are agreee and if something is wrong in the patch. Thank you for your hard and good work.

As soon as I've done some tests quickly, IMDB_ [EN] [HTTPS] script works fine and can be added to the new MOD version.

Notes for Ivek23:
I have tried to include all the fixes and improvements in this patch except the alternative scripts, for example IMDB_[EN][HTTPS]_(Cast&Crew) script.

OK.

I think that you are better positioned to decide the evolution of the script because I only use TheMovieDb and FilmAffinity, then in the future, please, change the versions of the main IMDB scripts (IMDB_[EN][HTTPS] and IMDB_People_[EN][HTTPS]) and I will include its in the MOD version same as you publish it.

I agree. Once the code changes and it will be necessary to fix it in the scripts, I will do it and post it on the forum. If there is a need to change something in scripts for better optimization of the scripts, you will also be welcome with your help.

Tell me if you are agreee and if something is wrong in the patch. Thank you for your hard and good work.

Thank you. I will continue to work as much as I can and it will be in my power.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on February 14, 2019, 12:15:59 pm
IMDB_People_[EN][HTTPS]

Why is not an added code for alternative names for people?
Why is not an added code for filmography for people?

If the code for both of them is added, IMDB_People_ [EN] [HTTPS] script will download all the information and will be the right and complete replacement for the IMDB People plugin.


I don't know …. I probably got lost among the different post of modifications. Sorry.
Can you help me by indicating the code to add or pointing to the posts where they are?

Or if the people script with the "+" of the post:  http://www.videodb.info/forum_en/index.php/topic,4134.msg20878.html#msg20878  (http://www.videodb.info/forum_en/index.php/topic,4134.msg20878.html#msg20878)
is the last and good versión I can publish in the MOD version adding one GET_FULL_FILMOGRAPHY parameter for the people that doesn't want download this information.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on February 14, 2019, 05:58:42 pm
IMDB_People_[EN][HTTPS] script

Or if the people script with the "+" of the post:  http://www.videodb.info/forum_en/index.php/topic,4134.msg20878.html#msg20878  (http://www.videodb.info/forum_en/index.php/topic,4134.msg20878.html#msg20878)
is the last and good versión I can publish in the MOD version adding one GET_FULL_FILMOGRAPHY parameter for the people that doesn't want download this information.

Yes, this is the latest good version, which works. You can add a GET_FULL_FILMOGRAPHY parameter. It starts at the end of the photo code and the beginning of the Award code.
Does not add with this part of the code:

Quote
(*
   //~Filmography Composer - Soundtrack~
   If Pos('<h2>Filmography</h2>', HTML) > 0 Then Begin
   curPos:=Pos('<h2>Filmography</h2>',HTML);
      If curPos>0 Then Begin
         EndPos:=curPos;
         curPos:=Pos('<h2>Filmography</h2>', HTML);
         curPos:=PosFrom('<a name="soundtrack">Soundtrack</a> ',HTML,curPos);
         curPos:=curPos+Length('<a name="soundtrack">Soundtrack</a> ');
         endPos:=PosFrom('<div id="filmo-head',HTML,curPos);
         if (EndPos < 1) OR (EndPos > PosFrom('</div>    <script>',HTML,curPos)) then
         endPos:=PosFrom('</div>    <script>',HTML,curPos);
         ItemList:=Copy(HTML,curPos,endPos-curPos);
         //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');
         curPos:=Pos('" id="soundtrack-tt',ItemList);
            Year:=TextBetWeen(ItemList,'<span class="year_column">','</span>',false,curPos);
            LogMessage('      Parse Results Year:'+Year+'||');
            If curPos>0 Then Begin
            curPos:=Pos('<b><a href="/title/',ItemList);       
            index:=1;
            While curPos>0 Do Begin
               //MovieURL:=TextBetWeen(ItemList,'<b><a href="/title/','?ref_=nm_flmg_act',false,curPos);
               MovieURL:='http://www.imdb.com/title/'+TextBetWeen(ItemList,'<b><a href="/title/','?ref_=_',false,curPos);
               LogMessage('      Parse Results MovieURL:'+MovieURL+'||');
               Title:=TextBetWeen(ItemList,'>','</a></b>',false,curPos);
               LogMessage('      Parse Results Title:'+Title+'||');            
               AddPersonMovie(Title,'','',Year,MovieURL,ctComposers);
               LogMessage('      Get results Composers:#'+IntToStr(index)+'|'+Title+'|'+MovieURL+'||ctComposers');
               Year:=TextBetWeen(ItemList,'<span class="year_column">','</span>',false,curPos);
               LogMessage('      Parse Results Year:'+Year+'||');            
               curPos:=PosFrom('<b><a href="/title/',ItemList,curPos);   
               index:=index+1;
            End;   
            curPos:=Pos('" id="soundtrack-tt',ItemList);
         End;
      End;
   End;  *)
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on February 17, 2019, 09:21:28 pm
Attached temporal Acumulative Scripts Patch date: 2019.02.17 (to copy in PVD folder for overwriting the existing files).

Changelog:
        IMDB_People_[EN][HTTPS]
                 V 1.3.2.0 (17/02/2019) Ivek23: Added saving more infos: Filmography, Alternate Names, etc. Set by default PHOTO-URL-IN-TRANSNAME=False

Notes for Ivek23:
GET_FULL_FILMOGRAPHY parameter is not needed because PVD has it in the 'Set Overwrite Options for the selected plugin'.
I'd tried Alternate Name with Charles Chaplin  https://www.imdb.com/name/nm0000122/ (https://www.imdb.com/name/nm0000122/) and the script doesn't got it (even looking in the Log window).
If you find the good code, please upload it with next version number 1.3.3.0
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on February 22, 2019, 07:54:41 am
Notes for Ivek23:
GET_FULL_FILMOGRAPHY parameter is not needed because PVD has it in the 'Set Overwrite Options for the selected plugin'.

Exactly, it's true.

Notes for Ivek23:
I'd tried Alternate Name with Charles Chaplin  https://www.imdb.com/name/nm0000122/ (https://www.imdb.com/name/nm0000122/) and the script doesn't got it (even looking in the Log window).

It worked without a problem with me. I added another part of the new code, which also works properly.
The old part of the code that works with me
Quote
   //~Alternate Names~
   curPos:=Pos('<div id="details-akas" class="see-more inline canwrap">',HTML);
   If curPos>0 Then Begin
      EndPos:=curPos;
      ItemValue1:=HTMLValues2(HTML,'<h4 class="inline">Alternate Names:','</div>','</h4>','</div>','<br>',EndPos);
      //ItemValue1:=HTMLValues(HTML,'Alternate Names:','</div>','</h4>','</div>','<br>',EndPos); //
      //ItemValue1:=TextBetween(HTML,'<h4 class="inline">Alternate Names:</h4>','</div>',True,curPos);
      LogMessage('    *    Parse Results AltNames01:'+ItemValue1+'||');
      ItemValue1:=StringReplace(ItemValue1,' | ','<br>',True,False,True);
      //ItemValue1:=StringReplace(ItemValue1,' | ',#13,True,False,True);   
      //if Pos(', ', ItemValue1) = 1 then Delete(ItemValue1,1,2);
      If ItemValue1 <> '' then AddFieldValueXML('AltNames',ItemValue1);
      LogMessage('      Parse Results AltNames02:'+ItemValue1+'||');
   End;
The new part of the code, which also works with me
Quote
   //~Alternate Names~
   EndPos:=PosFrom('<div id="details-akas" class="see-more inline canwrap">',HTML,EndPos);
   curPos:=PosFrom('<h4 class="inline">Alternate Names:</h4>',HTML,EndPos);
   EndPos:=PosFrom('</div>',HTML,curPos);
   ItemValue:=TextBetween(HTML,'<h4 class="inline">Alternate Names:</h4>','</div>',True,curPos);
   LogMessage('    *    Parse Results AltNames0:'+ItemValue+'||');
   ItemValue:=StringReplace(ItemValue,' | ','<br>',True,False,True);
   //ItemValue:=StringReplace(ItemValue,' | ',',  ',True,False,True);
   //If Pos(', ',ItemValue) = 1 then Delete(ItemValue,1,2);
   If ItemValue <> '' then AddFieldValueXML('AltNames',ItemValue);
   LogMessage('      Parse Results AltNames:'+ItemValue+'||');

Notes for Ivek23:
If you find the good code, please upload it with next version number 1.3.3.0

A new version of IMDB_People_ [EN] [HTTPS] script will be uploaded in a few days, at least I hope it will. I do not have much time to test IMDB_People_ [EN] [HTTPS] script, if everything works fine because I found some code errors for filmography of people.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on February 23, 2019, 08:18:11 am
Thank you, Ivek23.
I stay tuned in order to update with it the MOD version.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on March 01, 2019, 08:54:27 pm
IMDB_People_ [EN] [HTTPS] script
    V 1.3.3.0 (01/03/2019)


Some code corrections:
Quote
Function DownloadPage(URL:AnsiString):String; //BlockOpen
//Returns the URL page text. If error returns empty string
  Var
    i:Integer;
    ScriptPath,WebText:String;
    Begin
.
.
.
         i:=i+1;
         If i=INTERNET_TEST_ITERATIONS Then Begin
            if 2=MessageBox('Too many faulty attempts to internet connection.'+Chr(13)+ 'Retry or Cancel?',SCRIPT_NAME,5) then begin
               LogMessage(Chr(9)+Chr(9)+'Function DownloadPage END with NOT INTERNET connection ===============|');
               Result:='';
               Exit;
            End;
            i:=0;
         End;
.
.
.
    if (Pos('405 Method not allowed',Result)>0) then begin  //Banned IP. WEB_SPECIFIC.
        If BYPASS_SILENT Then ShowMessage('The URL has HTTP method problems (405 Method not allowed).'+Chr(13)+'Go to the provider web in order to in order to'+Chr(13)+'find the good page',SCRIPT_NAME);
        LogMessage(Chr(9)+Chr(9)+'      405 Method not allowed|');
        Result:=''; //Fatal error
    End;
    if (Pos('Too many request',Result)>0) then begin  //Banned IP. WEB_SPECIFIC.
        If BYPASS_SILENT Then ShowMessage('The provider has banned your IP (Too many request).'+Chr(13)+'Go to the provider web and resolve the captcha in order to'+Chr(13)+'demostrate you are not a robot',SCRIPT_NAME);
        LogMessage(Chr(9)+Chr(9)+'      Banned IP|');
        Result:=''; //Fatal error
    End;
    LogMessage(Chr(9)+Chr(9)+'Function DownloadPage END======================|');
    exit;
  End; //BlockClose
Function DownloadImage

Lots of fixes of Filmography code and added code for Genres links in the Comment box.
Quote
   //Get ~Genre index URL~
   //http://www.imdb.com/filmosearch?sort=moviemeter&explore=genres&role=nm0000122&ref_=nm_ql_flmg_5
   EndPos:=Pos('class="link" ><span class="ghost">by</span> Genre</a>',HTML);
   If endPos>0 Then Begin
      ItemValue0:='<link url="http://www.imdb.com/filmosearch?sort=moviemeter&explore=genres&role='+PersonID+'&ref_=nm_ql_flmg_5">Genres</link>';
      If ItemValue0 <> '' then ItemList:=ItemList+#13+ItemValue0;
      LogMessage('      Parse Results Genre URL:'+ItemValue0+'||');
   End;

A new code has also been added to download information for genres in the Genres field.

Quote
//Retreive Data Config
  USE_SAVED_PVDCONFIG  = True ;  //Use the Overwrite Options of the script saved in pvdconf.ini for avoid download not used pages. Remember PVD only save in exit.
  GET_FULL_BIO  = True ;  //Download Biography provider page for retreive the info. Otherwise only the info of the principal movie page.
   GET_FULL_GENRES  = True ;  //Download Genres provider page for retreive the info. Otherwise only the info of the principal movie page.
Quote
Function ParsePage_IMDBPeopleGENRES(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit;
    //Retrieve: ~genres~ Genres from "Genres" IMDB section
  Var
    curPos,endPos:Integer;
    ItemValue:String;
  Begin
    LogMessage('Function ParsePage_IMDBPeopleGENRES BEGIN=====================||');       
    Result:=prFinished;  //It will change to prError if any big problem with exit;
   //http://www.imdb.com/filmosearch?sort=moviemeter&explore=genres&role=nm0000122&ref_=nm_ql_flmg_5
   //Get ~Genres~
   If Pos('<strong>Genres</strong>',HTML)>0 Then Begin
      curPos:=Pos('<strong>Genres</strong>',HTML);
      EndPos:=curPos;
      ItemValue:=HTMLValues2(HTML,'<fieldset data-join="and" name="genres">','</fieldset>','<span class="faceter-facet-text">','</span>',', ',EndPos);            
      AddFieldValueXML('genre',ItemValue);    
      curPos:=EndPos;
      LogMessage('      Get result Genres (from People Genres):'+ItemValue+'||');
   End;   
    LogMessage('Function ParsePage_IMDBMovieGENRES END=====================||');
  End; //BlockClose

Minor correction also Bio Codes.

IMDB_People_ [EN] [HTTPS] script is attached.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: webmasterone on March 06, 2019, 03:25:26 pm
Hallo
first of all, thanks for this grat job
i discovered it only now
i used pdv in the past, but were passed to "Ant Movie Catalog" when pvd stopped to work corrrectly

i would like to make you 2 question:

1) is there a localized version of PVD in Italian ? if not, is there a way to contribute to create it ?
2) i would like to have the search script work for italian movie (so to add also [IT] language to script, but i don't know what to do.

can you kindly help me ?
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on March 07, 2019, 08:22:20 am
Hallo
first of all, thanks for this grat job
i discovered it only now

Thanks for the name of the program's author and others, who help the program continue to work in some way.

1) is there a localized version of PVD in Italian ? if not, is there a way to contribute to create it ?

If you use 0.9.9.21 version, it already has an Italian translation. V1.0.2.7 does not have an Italian translation. However, there is a possibility to translate the english.rc file into the Italian language and rename it to custom.lng and copy it to the program folder. Now PVD will be in your language.

More info:
http://www.videodb.info/forum_en/index.php/topic,3186.msg15802.html#msg15802


The english.rc file is attached.


2) i would like to have the search script work for italian movie (so to add also [IT] language to script, but i don't know what to do.

In the script, add the following as shown below
Quote
  SCRIPT_LANG  = $09; //English //$0a es-Spanish   $0c fr-French   $09 en-English   $10 it-Italian
of course, if the website supports or is in the Italian language. A good example of this is TMDB web page (https://www.themoviedb.org/).

There is currently no operating script for the PVD MOD version in the Italian language.

Perhaps what works in v0.9.9.2.1 and v1.0.2.7 with the help of
Proxomitron program (http://www.videodb.info/forum_en/index.php/topic,4083.0.html), where it is necessary to add certain web pages in default.cfg, so that everything works together.

Write the addresses and links for which web pages you need, then we will then see what can be edited and helped.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: webmasterone on March 07, 2019, 01:53:49 pm
Ok
i'm translating language file
as soon as i will end it, i will give it to you, so that you can include it also in the installation files

I copied the [EN] file and renamed it TheMovieDB_[IT][API].psf
Than I modified the file as you told me (  SCRIPT_LANG  = $10; //$07=DE-German, $09=EN-English, $0a=ES-Spanish, $0c=FR-French, $10 it-Italian )
but i noted that the description still remained in english
i found that after modifying (in the same file) also the 128 line
  LANG_API_STR    = '&language=it'; //'&language=en', '&language=de', '&language=en', '&language=es', '&language=fr' //WEB_SPECIFIC
the description of the movie is now written in italian

just another thing
i read in the script that:
"TheMovieDB "keywords" info goes to ~tags~ PVD field"
but i can see that all the tags (TheMovieDB Keywords) are still in english (see image attached)

can you suggest to me how to solve it ?

thanks




Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on March 07, 2019, 03:44:07 pm
Nice work and congratulations that something has just succeeded.

Ok
i'm translating language file
as soon as i will end it, i will give it to you, so that you can include it also in the installation files.

You do not send it to me, because I can not add it to the installation files. Attach it (in 7z or zip formats) to a forum in this topic for other users, which is then added as custom.lng to its PVD program folder.

just another thing
i read in the script that:
"TheMovieDB "keywords" info goes to ~tags~ PVD field"
but i can see that all the tags (TheMovieDB Keywords) are still in english (see image attached)

can you suggest to me how to solve it ?

It will probably be everywhere because it is not translated into the Italian language. Unfortunately, this can not be corrected. There is, however, the possibility that it will be OK when the website is also translated into the Italian language.
Title: Re: Possibles improving Custom Fields working in MOD version
Post by: afrocuban on March 07, 2019, 09:55:45 pm
Hello, and thank you, thank you, thank you for your great work, guys!

I wanted to ask you if it's possible to include custom fields "IMDb.com", "AllMovie.com", "Rottentomatoes.com" and "FilmAffinity.com" in the scripts?

Thank you in advance!
Title: Re: Possibles improving Custom Fields working in MOD version
Post by: Ivek23 on March 08, 2019, 09:22:23 pm
Hello, and thank you, thank you, thank you for your great work, guys!

I wanted to ask you if it's possible to include custom fields "IMDb.com", "AllMovie.com", "Rottentomatoes.com" and "FilmAffinity.com" in the scripts?

Thank you in advance!

Of course, it depends on why you will need them, or for what purpose is the transfer of information that you need (what you will download to these custom fields).
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on March 08, 2019, 09:36:26 pm
Quote from: Ivek23
Attach it (in 7z or zip formats) to a forum in this topic for other users, which is then added as custom.lng to its PVD program folder.
This is a good idea: If people attach here his Custom.lng files I'll include its in the MOD version (even perhaps I can do a little batch program in order to ask to user and tralate PVD auto).
Quote from: webmasterone
I copied the [EN] file and renamed it TheMovieDB_[IT][API].psf
Than I modified the file as you told me (  SCRIPT_LANG  = $10; //$07=DE-German, $09=EN-English, $0a=ES-Spanish, $0c=FR-French, $10 it-Italian )
but i noted that the description still remained in english
i found that after modifying (in the same file) also the 128 line
  LANG_API_STR    = '&language=it'; //'&language=en', '&language=de', '&language=en', '&language=es', '&language=fr' //WEB_SPECIFIC
the description of the movie is now written in italian
Well done! You have found the good solution.
And perhpas more ... the diferent language versions of the TheMovieDB scripts are automatic translations using the attached files. If you give me the italian file I'll traslate it and include in the version MOD
Quote from: webmasterone
i read in the script that:
"TheMovieDB "keywords" info goes to ~tags~ PVD field"
but i can see that all the tags (TheMovieDB Keywords) are still in english (see image attached)
Ivek23 has reason: This information is in english in the TheMovieDB website for the films in all languages. In the other hand the Genre are normaly translated.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: VVV_Easy_Programing on March 08, 2019, 10:21:26 pm
Changelog:
Last version always in the first post
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on March 16, 2019, 09:11:59 am
Quote from: VVV_Easy_Programing
Quote from: Ivek23
Attach it (in 7z or zip formats) to a forum in this topic for other users, which is then added as custom.lng to its PVD program folder.
This is a good idea: If people attach here his Custom.lng files I'll include its in the MOD version (even perhaps I can do a little batch program in order to ask to user and tralate PVD auto).

Slovenian.lng file for PVD and PVD MOD 1.0.2.7 version you will find on this link:

http://www.videodb.info/forum_en/index.php/topic,1752.msg20918.html#msg20918
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: afrocuban on March 17, 2019, 11:43:02 pm
Hello, and thank you, thank you, thank you for your great work, guys!

I wanted to ask you if it's possible to include custom fields "IMDb.com", "AllMovie.com", "Rottentomatoes.com" and "FilmAffinity.com" in the scripts?

Thank you in advance!

Of course, it depends on why you will need them, or for what purpose is the transfer of information that you need (what you will download to these custom fields).

Thank you in advance.

I use them, for example:

Code: [Select]
//Get ~mfURL~ or ~IMDb Url~
endPos := Pos('/ "><meta property="og:url"', HTML);
if endPos > 0 then begin
curPos := PrevPos('"canonical" href="', HTML, endPos);
AddFieldValue(mfURL, Copy(HTML, curPos + 18, endPos - curPos - 18));
end else
AddFieldValue(mfURL, MovieURL);
AddCustomFieldValueByName('IMDb.com', '<link url="' + MovieURL + '">IMDb.com</link>');
TmpStr := StringRepl


Just to get URL in a form:

<link url="http://www.allmovie.com/movie/upgrade-v699248">AllMovie.com</link>

But, this code above only works in some old IMDb script and Allmovie_new_HTTPS.psf with Proxomitron, and Rottentomatoes HTTPS script without Proxomitron, but I always have to edit them after their update/upgrade to insert those custom fields...

Thank you in advance
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on March 21, 2019, 08:12:29 am
We will add and edit this in the following updates.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on April 14, 2019, 07:16:15 am
We will add and edit this in the following updates.
Will this be arranged with the next update, but it will be necessary to make for each script its own custom field to save the url of the address. Each update in this field changes the record and needs to be corrected. Also, in the scripts, it will be necessary to add this part of the code to the custom field.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: jippo on June 16, 2019, 10:36:52 pm
I'm having this since two weeks ago.

I'm scanning with IMDB.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on June 18, 2019, 03:44:12 pm
I'm having this since two weeks ago.

I'm scanning with IMDB.

Which IMDB version of the script, or which IMDB script you are using. Which film title is problematic for this error. We need the name and url of the movie title to help solve the problem.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: jondak on June 19, 2019, 06:46:27 pm
I'm having this since two weeks ago.

I'm scanning with IMDB.

Same error:
IMDB_[EN][HTTPS].psf V 1.4.1.0 (10/02/2019)

For now 2 movies triggered the error:
https://www.imdb.com/title/tt1210059/ Flying Lessons (2010)

https://www.imdb.com/title/tt1877647/ Ghoul (2012)

Seems the movies with this error have no Plot Keywords

Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on June 20, 2019, 08:19:06 am
I'm having this since two weeks ago.

I'm scanning with IMDB.

Same error:
IMDB_[EN][HTTPS].psf V 1.4.1.0 (10/02/2019)

For now 2 movies triggered the error:
https://www.imdb.com/title/tt1210059/ Flying Lessons (2010)

https://www.imdb.com/title/tt1877647/ Ghoul (2012)

Seems the movies with this error have no Plot Keywords

Thanks for the error message. It's true, what you have already mentioned, that your movie titles have no Plot Keywords. I corrected this error.

Quote
CHANGE LOG :
            V 1.4.1.1 (20/06/2019) Ivek23: ParsePage_PLOTKEYWORDS small corrections and fix in the parts of the code 18-19/6/2019 forum alerts.
.
.
.
Function ParsePage_IMDBMoviePLOTKEYWORDS(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If ¿any big problem? with exit
    //Retrieve: ~Tags~
  Var
      curPos,endPos,index:Integer;
      //curPos,endPos:Integer;
      //ItemValue:String;
     ItemValue,ItemList:String;
     //Category,URL,Name:String;
     Category,Name:String;
     ItemArray:TWideArray;    
  Begin
    LogMessage('Function ParsePage_IMDBMoviePLOTKEYWORDS BEGIN======================|');
    Result:=prFinished;  //It will change to prError if any big problem with exit;   
   //Get "Plot Keywords" info
    curPos:=Pos('<h1 class="header">Plot Keywords</h1>',HTML);
    if (curPos=0) then Exit;
    EndPos:=curPos;   
   //Get "Plot Keywords" info
   //If Pos('<h1 class="header">Plot Keywords</h1>',HTML)>0 Then Begin
   curPos:=Pos('<h1 class="header">Plot Keywords</h1>',HTML);
   If 0<curPos Then Begin   
      EndPos:=curPos;   
      Category:=TextBetween(HTML,'<h1 class="header">','</h1>',True,curPos);
      LogMessage('      Get result Category:'+Category+'||');
      Name:=TextBetween(HTML,'<div class="header"><div class="nav"><div class="desc">','</div></div></div>',True,curPos);
      LogMessage('      Get result Name:'+Name+'||');
      AddCustomFieldValueByName('Plot Keywords','<link url="'+GetFieldValueXML('url')+'keywords">'+Category+'</link>'+'     '+Name);
   End;         
   //Get "Plot Keywords" info
If (Pos('It looks like we don'+Chr(39)+'t have any Plot Keywords for this title yet.',HTML)>0) then Exit;
   curPos:=Pos('<h1 class="header">Plot Keywords</h1>',HTML);
   If 0<curPos Then Begin   
      //EndPos:=curPos;
      //Go Tags list
      curPos:=Pos('<table class="dataTable evenWidthTable2Col"><tbody>',HTML);
      curPos:=curPos+Length('<table class="dataTable evenWidthTable2Col"><tbody>');
      endPos:=PosFrom('</table>',HTML,curPos);
      //ItemList:=Copy(HTML,curPos,endPos-curPos);
      ItemList:=Trim(Copy(HTML,curPos,endPos-curPos));
      curPos:=Pos('tp://ogp.me/ns#"',ItemList);
        If 0<curPos then ItemList:=Copy(ItemList,0,curPos-1);
      ItemList:=RemoveTagsEx(ItemList);      
      ItemList:=StringReplace(ItemList,'    </div>','|',True,True,False);               
      ItemList:=RemoveTags(ItemList, False);
      //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||'+#13);         
      //LogMessage('           Parse results ItemList:'+#13+ItemList+' ||');
      //LogMessage('           Parse results ItemList: '+ItemList+' ||');
      ExplodeString(ItemList,ItemArray,'|');
      For index:=Low(ItemArray) To High(ItemArray) Do Begin        //Remember index begin in 0
            If (index=PLOTKEYWORDS_LIMIT) Then break;     //Limited depassed. Compares with = (not with >) because index begin in 0
         ItemValue:=ItemValue+ItemArray[index]+', ';
         LogMessage('    Get result Keyword: #'+IntToStr(index+1)+' | '+ItemArray[index]+', '+'||');
      End;
      index:=index+1;      
      If ItemValue <> '' then AddFieldValueXML('tags',ItemValue);
      LogMessage('      Get result Plot Keywords: '+ItemValue+'||');
    End;   
    LogMessage('Function ParsePage_IMDBMoviePLOTKEYWORDS END=====================||');
  End; //BlockClose

It's attached IMDB_ [EN] [HTTPS] V 1.4.1.1 script.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: jippo on June 23, 2019, 10:41:46 pm
Thanks.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: jondak on July 02, 2019, 11:08:53 am
Thank you.
Title: Re: Personal Video Database 1.0.2.7 MOD
Post by: Ivek23 on July 21, 2019, 06:24:12 am
I'm having this since two weeks ago.

I'm scanning with IMDB.

Same error:
IMDB_[EN][HTTPS].psf V 1.4.1.0 (10/02/2019)

For now 2 movies triggered the error:
https://www.imdb.com/title/tt1210059/ Flying Lessons (2010)

https://www.imdb.com/title/tt1877647/ Ghoul (2012)

Seems the movies with this error have no Plot Keywords

Thanks for the error message. It's true, what you have already mentioned, that your movie titles have no Plot Keywords. I corrected this error.

Quote
CHANGE LOG :
            V 1.4.1.1 (20/06/2019) Ivek23: ParsePage_PLOTKEYWORDS small corrections and fix in the parts of the code 18-19/6/2019 forum alerts.
.
.
.
Function ParsePage_IMDBMoviePLOTKEYWORDS(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If ¿any big problem? with exit
    //Retrieve: ~Tags~
  Var
      curPos,endPos,index:Integer;
      //curPos,endPos:Integer;
      //ItemValue:String;
     ItemValue,ItemList:String;
     //Category,URL,Name:String;
     Category,Name:String;
     ItemArray:TWideArray;    
  Begin
    LogMessage('Function ParsePage_IMDBMoviePLOTKEYWORDS BEGIN======================|');
    Result:=prFinished;  //It will change to prError if any big problem with exit;   
   //Get "Plot Keywords" info
    curPos:=Pos('<h1 class="header">Plot Keywords</h1>',HTML);
    if (curPos=0) then Exit;
    EndPos:=curPos;   
   //Get "Plot Keywords" info
   //If Pos('<h1 class="header">Plot Keywords</h1>',HTML)>0 Then Begin
   curPos:=Pos('<h1 class="header">Plot Keywords</h1>',HTML);
   If 0<curPos Then Begin   
      EndPos:=curPos;   
      Category:=TextBetween(HTML,'<h1 class="header">','</h1>',True,curPos);
      LogMessage('      Get result Category:'+Category+'||');
      Name:=TextBetween(HTML,'<div class="header"><div class="nav"><div class="desc">','</div></div></div>',True,curPos);
      LogMessage('      Get result Name:'+Name+'||');
      AddCustomFieldValueByName('Plot Keywords','<link url="'+GetFieldValueXML('url')+'keywords">'+Category+'</link>'+'     '+Name);
   End;         
   //Get "Plot Keywords" info
If (Pos('It looks like we don'+Chr(39)+'t have any Plot Keywords for this title yet.',HTML)>0) then Exit;
   curPos:=Pos('<h1 class="header">Plot Keywords</h1>',HTML);
   If 0<curPos Then Begin   
      //EndPos:=curPos;
      //Go Tags list
      curPos:=Pos('<table class="dataTable evenWidthTable2Col"><tbody>',HTML);
      curPos:=curPos+Length('<table class="dataTable evenWidthTable2Col"><tbody>');
      endPos:=PosFrom('</table>',HTML,curPos);
      //ItemList:=Copy(HTML,curPos,endPos-curPos);
      ItemList:=Trim(Copy(HTML,curPos,endPos-curPos));
      curPos:=Pos('tp://ogp.me/ns#"',ItemList);
        If 0<curPos then ItemList:=Copy(ItemList,0,curPos-1);
      ItemList:=RemoveTagsEx(ItemList);      
      ItemList:=StringReplace(ItemList,'    </div>','|',True,True,False);               
      ItemList:=RemoveTags(ItemList, False);
      //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||'+#13);         
      //LogMessage('           Parse results ItemList:'+#13+ItemList+' ||');
      //LogMessage('           Parse results ItemList: '+ItemList+' ||');
      ExplodeString(ItemList,ItemArray,'|');
      For index:=Low(ItemArray) To High(ItemArray) Do Begin        //Remember index begin in 0
            If (index=PLOTKEYWORDS_LIMIT) Then break;     //Limited depassed. Compares with = (not with >) because index begin in 0
         ItemValue:=ItemValue+ItemArray[index]+', ';
         LogMessage('    Get result Keyword: #'+IntToStr(index+1)+' | '+ItemArray[index]+', '+'||');
      End;
      index:=index+1;      
      If ItemValue <> '' then AddFieldValueXML('tags',ItemValue);
      LogMessage('      Get result Plot Keywords: '+ItemValue+'||');
    End;   
    LogMessage('Function ParsePage_IMDBMoviePLOTKEYWORDS END=====================||');
  End; //BlockClose

It's attached IMDB_ [EN] [HTTPS] V 1.4.1.1 script.

IMDB_ [EN] [HTTPS] The 1.4.1.1 script is also available via the program's auto-update system.