Author Topic: Personal Video Database 1.0.2.7 MOD  (Read 5595 times)

0 Members and 1 Guest are viewing this topic.

Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2142
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #140 on: November 28, 2018, 09:02:33 am »
IMDB_ [EN] [HTTPS] (Plot Keywords) script

IMDB_ [EN] [HTTPS] (Plot Keywords) script now has the option to download the full number of Plot Keywords records. Also, the possibility of a limited number of Plot Keywords records is added, because in some cases a very large number of records is also available for a specific movie (the numbers also range over 400 or even 500 or more records). The limit is set to the number of 100 records in the PVD Tags field. IMDB_ [EN] [HTTPS] (Plot Keywords) script still transmits the full number of records, after which the filter allows 100 records in the PVD Tags field. Thus, IMDB_ [EN] [HTTPS] (Plot Keywords) script in some cases requires a lot of time to add records to the PVD database. I am still looking for a way or solution so that only a limited number of records (100) can be transmitted by IMDB_ [EN] [HTTPS] (Plot Keywords) script and written in the PVD Tags field and therefore it would take less time to add records to the PVD database .

Quote
  GET_FULL_PLOTKEYWORDS  = True ;  //Download Plot Keywords provider page for retreive the Plot Keywords info. Otherwise only the info of the principal movie page
  GET_PLOTKEYWORDS_LIMIT  = True ;  //Download Full of number of tags retrieved. Otherwise True for the only the Limit of number of tags retrieved.      * Get Limit of number of tags retrieved.
//  GET_PLOTKEYWORDS_LIMIT  = False ;  //Download Full of number of tags retrieved. Otherwise True for the only the Limit of number of tags retrieved.   ** Get Full of number of tags retrieved.
  PLOTKEYWORDS_LIMIT  = 100;    //Limit of number of tags retrieved.

In IMDB_ [EN] [HTTPS] (Plot Keywords) script, you can manually modify the way you transfer your records by blocking the current open setting and unblocking the currently blocked setting. With this change, the script will download the full number of records.

The new version of IMDB_ [EN] [HTTPS] (Plot Keywords) script is attached.
Ivek23
Win 7 32bit, 64bit   PVD v0.9.9.21, PVD v1.0.2.7 + MOD


Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2142
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #141 on: December 01, 2018, 03:34:45 pm »
IMDB_ [EN] [HTTPS] (Plot Keywords) script

Now there is the final version of IMDB [EN] [HTTPS] (Plot Keywords) script for use.

Changes to the final version of IMDB [EN] [HTTPS] (Plot Keywords) script:

Quote
  GET_FULL_PLOTKEYWORDS  = True ;  //Download Plot Keywords provider page for retreive the Plot Keywords info. Otherwise only the info of the principal movie page
  GET_PLOTKEYWORDS_LIMIT  = True ;  //Download Full of number of tags retrieved. Otherwise True for the only the Limit of number of tags retrieved.      * Get Limit of number of tags retrieved.
//  GET_PLOTKEYWORDS_LIMIT  = False ;  //Download Full of number of tags retrieved. Otherwise True for the only the Limit of number of tags retrieved.   ** Get Full of number of tags retrieved.
  PLOTKEYWORDS_LIMIT  = 100;    //Limit of number of tags retrieved.

The change is now described correctly and now looks like this:
Quote
  GET_FULL_PLOTKEYWORDS  = True ;  //Download Plot Keywords provider page for retreive the Plot Keywords info. Otherwise only the info of the principal movie page
  GET_PLOTKEYWORDS_LIMIT  = True ;    //Download Limit of number of tags retrieved.   * Get True for Limit of number of tags retrieved.
//  GET_PLOTKEYWORDS_LIMIT  = False ;    //Download Full of number of tags retrieved.  ** Get False for Full of number of tags retrieved.
  PLOTKEYWORDS_LIMIT  = 100;    //Limit of number of tags retrieved.
The download limit is set to 100 tags, but the script transfers 101 tags because the index starts to count at 0 digits.
In IMDB_ [EN] [HTTPS] (Plot Keywords) script, you can manually modify the way you transfer your records by blocking the current open setting and unblocking the currently blocked setting. With this change, the script will download the full number of records.

Added Plot Keywords to download info on the main movie page.

Now, the script works to limit the number of records faster, because I found a way that is faster than the previous one and only transfers as many records as the actual number is written in PLOTKEYWORDS_LIMIT options.

Quote
Function TextBetWeenFirst(_HTML:string;_ABegin,_AEnd:String):String; //BlockOpen
    //Returns the first extract part of a given _HTML between _Abegin and _AEnd and removes all tags from the result substring.
    //In order to avoid set the Pos variable in TextBetween
  Var
    _Pos:Integer;
  Begin
    _Pos:=0;
    Result:=TextBetWeen(_HTML,_ABegin,_AEnd,false,_Pos);
    Exit;   
End; //BlockClose 
function RemoveTagsEx(AText:String):String; //BlockOpen
Var
   B,E:Integer;
Begin
   Result:=AText;
   B:=PosFrom('    <div class="did-',Result,18);
   E:=PosFrom('      </span>',Result,B);
   While (B>0) AND (B<E) Do Begin
      Delete(Result,B,E-B+18);
      B:=Pos('    <div class="did-',Result);
      E:=Pos('      </span>',Result);
   End;
End; //BlockClose

Function ParsePage_BingSearch(HTML:String):Cardinal; //BlockOpen

This part of the code is also needed, otherwise the script does not work.

Also added is the Plot Keywords custom field in which the URL is transferred to the website and how much is the full number of Plot Keywords records for a specific movie title on the Plot Keywords web page. You can Plot Keywords custom field settings found in Possibles improving Custom Fields working in MOD version topic.

The final version of IMDB [EN] [HTTPS] (Plot Keywords) script is attached.
« Last Edit: December 02, 2018, 04:04:07 pm by Ivek23 »
Ivek23
Win 7 32bit, 64bit   PVD v0.9.9.21, PVD v1.0.2.7 + MOD


Offline VVV_Easy_Programing

  • Power User
  • ****
  • Posts: 184
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #142 on: December 02, 2018, 10:41:08 am »
Well done, Ivek32.
You really master the script working flow of the PVD MOD.
I'm going to integrate the retreive of Plotkeywords in the base script of IMDB (although 100 seem to me many) and Connections when you tell me it is ready.
Now I do not have much time, but it is first in my ToDo list.

Note: If Nostra wakes up and simply compiles the actual PVD with the libraries for HTTPS, we would have powerful scripts that could be quickly adapted to the oficial version.

Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2142
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #143 on: December 02, 2018, 04:08:43 pm »
Well done, Ivek32.
You really master the script working flow of the PVD MOD.
I'm going to integrate the retreive of Plotkeywords in the base script of IMDB (although 100 seem to me many) ...

Thank you.
 
I agree that there are 100 records too, so my suggestion is up to 50 records. I tend to be more inclined to a more appropriate number, that is, only 25 records in the database, what is the best variant that occupies a bit of space, especially when it is important in the record, which has a larger number of words in one record.

Note:
Quote
Function TextBetWeenFirst(_HTML:string;_ABegin,_AEnd:String):String; //BlockOpen
    //Returns the first extract part of a given _HTML between _Abegin and _AEnd and removes all tags from the result substring.
    //In order to avoid set the Pos variable in TextBetween
  Var
    _Pos:Integer;
  Begin
    _Pos:=0;
    Result:=TextBetWeen(_HTML,_ABegin,_AEnd,false,_Pos);
    Exit;   
End; //BlockClose 
function RemoveTagsEx(AText:String):String; //BlockOpen
Var
   B,E:Integer;
Begin
   Result:=AText;
   B:=PosFrom('    <div class="did-',Result,18);
   E:=PosFrom('      </span>',Result,B);
   While (B>0) AND (B<E) Do Begin
      Delete(Result,B,E-B+18);
      B:=Pos('    <div class="did-',Result);
      E:=Pos('      </span>',Result);
   End;
End; //BlockClose

Function ParsePage_BingSearch(HTML:String):Cardinal; //BlockOpen

This part of the code is also needed, otherwise the script does not work.

I'm going to integrate the retreive of Plotkeywords in the base script of IMDB (although 100 seem to me many) and Connections when you tell me it is ready.
Now I do not have much time, but it is first in my ToDo list.

Recommendation:
Yes, Plotkeywords in the base script of IMDB and Connections as a secondary script.


As soon as the final version of the Connections script is finished. There are still a few tests to work properly for the final version.

Note: If Nostra wakes up and simply compiles the actual PVD with the libraries for HTTPS, we would have powerful scripts that could be quickly adapted to the oficial version.

It would be great if your wish in the future will also come true, let's wait and we'll see ...
« Last Edit: December 02, 2018, 04:15:29 pm by Ivek23 »
Ivek23
Win 7 32bit, 64bit   PVD v0.9.9.21, PVD v1.0.2.7 + MOD


Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2142
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #144 on: December 15, 2018, 07:54:42 am »
IMDB_ [EN] [HTTPS] script

Minor changes to the Title / OrigTitle code because there were minor changes in the source code of the page. The correction will be added to the next version of the IMDB_ [EN] [HTTPS] script.


Quote
Function ParsePage_IMDBMovieBASE(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit;
    //Retrieve: ~title~, ~year~, ~origtitle~, ~poster~ / ~imdbrating~, ~IMDB_Votes~ (Custom Field) / ~TOP_250~(Custom Field) /
    //          If Not(GET_FULL_CREDIT): ~crew~ctDirectors,ctWriters,ctComposers,ctProducers(Not in base page), ctActors
    //         ~description~ / ~category~ "keywords" / ~tagline~ / ~genre~
    //         If Not(GET_FULL_MPAA) ~mpaa~
    //         ~country~ / ~rdate~ in contry provider local IP geolocation
    //         If Not(GET_FULL_AKA) ~aka~.
    //         ~budget~ / ~money~ / ~studio~ "Production Co"
    //         If Not(GET_FULL_FEATURES) ~features~
  Var
      curPos,endPos,index:Integer;
      ItemValue,ItemList,ImageFile:String;
      titleValue:String;
      Name,Role,PersonURL:String;
      ItemArray:TWideArray;
  Begin
    LogMessage('Function ParsePage_IMDBMovieBASE BEGIN======================|');
    Result:=prFinished;  //It will change to prError if any big problem with exit;
    //Because the script don't retreives the data in the page order a token search for the first curPos position or block select is mandatory
    //Get ~title~, ~year~, ~origtitle~, ~poster~
    //Get all "raw" title summary (in raw because we need the hidden links, we avoid "complete" token in strings which opens/closes)
    ItemList:=TextBetWeenFirst(HTML,'<div class="title_block"','div class="plot_summary_wrapper">'); //WEB_SPECIFIC.
    If (Length(ItemList)>0) Then Begin    
      //titleValue:=TextBetWeenFirst(ItemList,'<div class="title_wrapper"><h1 class="">','<');       //Strings which opens/closes the data. WEB_SPECIFIC
      titleValue:=TextBetWeenFirst(ItemList,'<h1 class="">','<');       //Strings which opens/closes the data. WEB_SPECIFIC
      If titleValue = '0' then titleValue:='';
      If titleValue = '' then titleValue:=TextBetWeenFirst(ItemList,'<h1 class="long">','<');       //Strings which opens/closes the data. WEB_SPECIFIC

      AddFieldValueXML('title',titleValue);
      LogMessage('      Get result title:'+titleValue+'||');
        ItemValue:=TextBetWeenFirst(ItemList,'<span id="titleYear">(',')</span>            </h1>');                                 //Strings which opens/closes the data. WEB_SPECIFIC      
        AddFieldValueXML('year',ItemValue);
        LogMessage('      Get result year:'+ItemValue+'||');
        ItemValue:=TextBetWeenFirst(ItemList,'<div class="originalTitle">','<');          //Strings which opens/closes the data. WEB_SPECIFIC
        If (Length(ItemValue)=0) Then ItemValue:=titleValue;                           //Provider hides the original title if same that title. WEB_SPECIFIC
        AddFieldValueXML('origtitle',ItemValue);
        LogMessage('      Get result origtitle:'+ItemValue+'||');
        ItemValue:=TextBetWeenFirst(ItemList,BASE_URL_IMAGE_PRE_TRUE,'.');                 //Get poster code. Strings which opens/closes the data. WEB_SPECIFIC
        If (Length(ItemValue)>0) then begin
            ItemValue:=BASE_URL_IMAGE_PRE_TRUE + ItemValue;                             //Base poster URL without '.jpg'. WEB_SPECIFIC
            ImageFile:=GetAppPath+'Scripts\'+BASE_DOWNLOAD_FILE_IMAGE_NAME+'-Poster.jpg'
            // Avoid HTTPS redirection: Download https image to file  OutPutFile=
            If (1=DownloadImage(ItemValue + '._V1_UY' + IntToStr(MAX_IMAGE_HEIGTH) + '_.jpg',ImageFile)) then begin  //Dowload in the selected user max size. WEB_SPECIFIC
                //AddImageURL(itPoster,ImageFile);    //Get the photo from a file even if the next line we log the true URL.
                LogMessage('      Get result poster:'+ItemValue + '._V1_UY' + IntToStr(MAX_IMAGE_HEIGTH) + '_.jpg'+'||');
            end else if (1=DownloadImage(ItemValue +'.jpg',ImageFile)) then begin  //Dowload in the web base size. WEB_SPECIFIC
                //AddImageURL(itPoster,ImageFile);    //Get the photo from a file even if the next line we log the true URL.
                LogMessage('      Get result poster:' + ItemValue + '.jpg'+'||');
            end;       
        end;
    End;
« Last Edit: December 15, 2018, 09:15:44 am by Ivek23 »
Ivek23
Win 7 32bit, 64bit   PVD v0.9.9.21, PVD v1.0.2.7 + MOD


 

anything