Personal Video Database

Thanks for the fast response dear Ivek. I have one question. Can your IMDB_[EN][HTTPS] _000a.psf script substitute all other IMDb scripts, except IMDB_[EN][HTTPS](episodes).psf of course?
Thank you for an answer in advance.

Thanks Ivek for the update.

Dear VVV,

Some of my URL fields probably had some special characters in IMDb addreses, because now after I manually deleted url, than import data from imdb, I got new imdb url in url field. To test this newly imported imdb url, I applied update again with "overwrite" option on and everything went well, so obviously you were right.

Thanks Ivek for the update.

I have discovered one more issue with both IMDB_[EN][HTTPS].psf and IMDB_[EN][HTTPS] _000.psf

I can update Series record (for example http://www.imdb.com/title/tt0863046/). But I can't update it's episodes. Updating just crashes PVD.

Here's part of the debug code

Code: [Select]

allocated memory  : 99,05 MB
command line      : viddb.exe -portable -debug
executable        : viddb.exe
exec. date/time   : 2018-08-08 10:01
version           : 1.0.2.7
compiled with     : Delphi 2010
madExcept version : 3.0l
callstack crc     : $0811da24, $53dbdf84, $4ecf9cfa
exception number  : 7
exception class   : Unknown
exception message : Unknown.

main thread ($9a4):
0811da24 +000 ???
008d0d77 +a27 viddb.exe    MainU    8520 +148 TPVDMain.ExecWebImport
008c5470 +3c8 viddb.exe    MainU    5445  +60 TPVDMain.DoPluginExecute
008cc7cf +057 viddb.exe    MainU    7482  +10 TPVDMain.ExecImpBtnClick
00551163 +06f viddb.exe    Controls           TControl.Click
005dc454 +000 viddb.exe    Buttons            TSpeedButton.Click
005dc43e +0ea viddb.exe    Buttons            TSpeedButton.MouseUp
00551598 +038 viddb.exe    Controls           TControl.DoMouseUp
00551614 +070 viddb.exe    Controls           TControl.WMLButtonUp
0055151e +07e viddb.exe    Controls           TControl.WMMouseMove
00550bf8 +2d4 viddb.exe    Controls           TControl.WndProc
0055081c +024 viddb.exe    Controls           TControl.Perform
00554de8 +0ac viddb.exe    Controls           TWinControl.IsControlMouseMsg
00555338 +3e4 viddb.exe    Controls           TWinControl.WndProc
00554b5c +02c viddb.exe    Controls           TWinControl.MainWndProc
004a9b5c +014 viddb.exe    Classes            StdWndProc
755f7885 +00a USER32.dll                      DispatchMessageW
005812c9 +11d viddb.exe    Forms              TApplication.ProcessMessage
0058130e +00a viddb.exe    Forms              TApplication.HandleMessage
00581639 +0c9 viddb.exe    Forms              TApplication.Run
009af241 +b69 viddb.exe    viddb     257 +120 initialization
75e03368 +010 kernel32.dll                    BaseThreadInitThunk

thread $1828 (TWorkerThread):
77ccf8da +0e ntdll.dll                           NtWaitForSingleObject
775015c8 +92 KERNELBASE.dll                      WaitForSingleObjectEx
75e0118f +3e kernel32.dll                        WaitForSingleObjectEx
75e01143 +0d kernel32.dll                        WaitForSingleObject
005a2651 +19 viddb.exe      VirtualTrees 6002 +3 TWorkerThread.Execute
00467507 +2b viddb.exe      madExcept            HookedTThreadExecute
004a703a +42 viddb.exe      Classes              ThreadProc
00406c38 +28 viddb.exe      System        985 +0 ThreadWrapper
004673e9 +0d viddb.exe      madExcept            CallThreadProcSafe
00467453 +37 viddb.exe      madExcept            ThreadExceptFrame
75e03368 +10 kernel32.dll                        BaseThreadInitThunk
>> created by main thread ($9a4) at:
005a2596 +16 viddb.exe      VirtualTrees 5965 +1 TWorkerThread.Create

thread $1e90:
77cd0166 +0e ntdll.dll     NtWaitForMultipleObjects
75e03368 +10 kernel32.dll  BaseThreadInitThunk

thread $1d80:
77ccf8da +0e ntdll.dll       NtWaitForSingleObject
775015c8 +92 KERNELBASE.dll  WaitForSingleObjectEx
75e0118f +3e kernel32.dll    WaitForSingleObjectEx
75e01143 +0d kernel32.dll    WaitForSingleObject
6a1a29b8 +38 MSVCR80.dll     _endthreadex
75e03368 +10 kernel32.dll    BaseThreadInitThunk

thread $1590: <priority:2>
77ccf8da +0e ntdll.dll       NtWaitForSingleObject
775015c8 +92 KERNELBASE.dll  WaitForSingleObjectEx
75e0118f +3e kernel32.dll    WaitForSingleObjectEx
75e01143 +0d kernel32.dll    WaitForSingleObject
6a1a29b8 +38 MSVCR80.dll     _endthreadex
75e03368 +10 kernel32.dll    BaseThreadInitThunk

thread $15ec:
77cd1f4f +0b ntdll.dll     NtWaitForWorkViaWorkerFactory
75e03368 +10 kernel32.dll  BaseThreadInitThunk

disassembling:
[...]
008d0d67        mov     eax, [$9cb0b8]
008d0d6c        mov     eax, [eax]
008d0d6e        mov     eax, [eax+$30]
008d0d71        call    -$4c988e ($4074e8)     ; System.@LStrToPChar
008d0d76        push    eax
008d0d77      > call    dword ptr [ebp-$18]
008d0d7a        movsx   esi, ax
008d0d7d 8521   sub     esi, 1
008d0d80        jb      loc_8d17aa
008d0d86        jz      loc_8d0d9d
008d0d88        dec     esi

I suspect it has something to do with double quotes  - ". When " is in title (for the episode, importing is tried from the page which title is series name under the double qoutes, then episode name, then PVD crashes.

For example, for the http://www.imdb.com/title/tt1006987/  ,
script tries to import data from imdb page which title is (can be seen on browser tab) "Flight of the Conchords": Sally, or something similar, but it definitely has double quotes,

Hopefully I can get some advise.

Quote from: VVV_Easy_Programing on October 03, 2018, 12:13:40 pm

BTW, I have included several 'hidden' Custom Fields of Ivek23 in the scripts.
Ivek23, perhaps can be useful for other users open a new Thread with the information "Possibles improving Custom Fields" working in MOD version, how is the information and how add in the PVD database.

To that, when I do some more tests, because I discovered some more errors and I still test some more improvements to the code sections for IMDB_ [EN] [HTTPS] script.

• Bottom 100
• Metascore
• User Reviews
• ImdbTechSpecs
• IMDbID

• Bottom 100 (Number)
• Metascore (Rating)
• User Reviews (Memo)
• ImdbTechSpecs (Memo)
• IMDbID (Short Text)

• IMDB Rating
• IMDB Release Date

• IMDB Rating (Rating)
• IMDB Release Date (Multiselect list)

• FilmAffinity [HTTPS] all version scripts
• TheMovieDB [API] all version scripts
• IMDB_[EN][HTTPS] script

• Updated
• FilmAffinity_Votes
• FilmAffinity_Rating

• Updated
• Set
• TheMoviedb_Votes

• Updated
• TOP_250
• IMDB_Votes

    //Date ~Updated~ (choose simple or verbose version)
        Date:=DateToStr(CurrentDateTime);
        //AddFieldValueXML('viewdate',Date); //Only date, don't admit time-. Set Seen value at the same time.
        //AddFieldValueXML('moddate',Date+' '+TimeToStr(CurrentDateTime)); //Block the dB saving   
        ExplodeString(Date,DateParts,'-');
        Date:=DateParts[2]+'.'+ DateParts[1]+'.'+DateParts[0]; 
        Date:=StringReplace(Date,'01.','1.',True,True,False);
      Date:=StringReplace(Date,'02.','2.',True,True,False);
      Date:=StringReplace(Date,'03.','3.',True,True,False);
      Date:=StringReplace(Date,'04.','4.',True,True,False);
      Date:=StringReplace(Date,'05.','5.',True,True,False);
      Date:=StringReplace(Date,'06.','6.',True,True,False);
      Date:=StringReplace(Date,'07.','7.',True,True,False);
      Date:=StringReplace(Date,'08.','8.',True,True,False);
      Date:=StringReplace(Date,'09.','9.',True,True,False);         
        //AddCustomFieldValueByName('Updated',Date); // Simple
        //AddCustomFieldValueByName('Updated',Date+' at '+TimeToStr(CurrentDateTime)); // Verbose
        AddCustomFieldValueByName('Updated',Date+' at '+TimeToStr(CurrentDateTime)+' • '+SCRIPT_NAME+' '+SCRIPT_VERSION);  // Annoying

Function ParsePage_IMDBMovieMPAA(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError;  If any big problem? with exit
    //Retrieve: ~MPAA~
  Var
    curPos:Integer;
    mpaaValue:String;
  Begin
.
.
.
    mpaaValue:=StringReplace(mpaaValue,'Suggested Rating','. Suggested Rating',False,False,False); //Avoid union in films with 'Suggested Rating'
   mpaaValue:=StringReplace(mpaaValue,'update: ','',False,False,True);                         //Cleanig value. //WEB_SPECIFIC
   mpaaValue:=StringReplace(mpaaValue,': ','',False,False,True);                         //Cleanig value. //WEB_SPECIFIC
    curPos:=Pos('Edit',mpaaValue);                                     //WEB_SPECIFIC.
    If 0<curPos then mpaaValue:=Copy(mpaaValue,0,curPos-1);   
    If (mpaaValue='Approved') Then mpaaValue:='Rated G';
.
.
.
    AddFieldValueXML('mpaa',mpaaValue);
    LogMessage('      Get result MPAA='+mpaaValue+'||');
    LogMessage('Function ParsePage_IMDBMovieMPAA END=====================||');
  End; //BlockClose

    //Go to "Directed by" ~crew~ctDirectors
    curPos:=Pos('<h4 class="dataHeaderWithBorder">Directed by',HTML);                     //Strings start which opens the block content data. WEB_SPECIFIC   
    If 0<curPos Then Begin
      curPos:=PosFrom('</h4>',HTML,curPos);                                              //Strings end which opens the block content data.  WEB_SPECIFIC
      curPos:=curPos+Length('</h4>');                                                    //Strings end which opens the block content data.  WEB_SPECIFIC
      //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~ n the ~crew~
      endPos:=PosFrom('</table>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
      ItemList:=Copy(HTML,curPos,endPos-curPos);
      //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');
      curPos:=Pos('<td class="name">',ItemList)                                         //String which opens the subList data. WEB_SPECIFIC
      index:=1;
      While curPos>0 Do Begin
         If (index>PEOPLE_LIMIT) Then break;     //Limited depassed (Remember index begin in 0).
         //Get PersonURL (Always must it has)
         PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF;     //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
         //Get Name (Always must it has)
         Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC     
         LogMessage('      Parse Results Name:'+Name+'||');
         //Get Role: PVD don't save Role in crew people but is added for Ivek23 as LogMessage compatibility
         //Role:=TextBetWeen(ItemList,'<td class="credit">','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC
            Role:=TextBetWeen(ItemList,'<td ','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC         
         LogMessage('      Parse Results Role:'+Role+'||');
         AddMoviePerson(Name,'','',PersonURL,ctDirectors);
         LogMessage('      Get results Director:#'+IntToStr(index)+'|'+Name+'|'+PersonURL+'||ctDirectors');
         curPos:=PosFrom('<td class="name">',ItemList,curPos)                                            //String which opens the subList data. WEB_SPECIFIC
         index:=index+1;
      End;
    End;

BTW, I have included several 'hidden' Custom Fields of Ivek23 in the scripts.
Ivek23, perhaps can be useful for other users open a new Thread with the information "Possibles improving Custom Fields" working in MOD version, how is the information and how add in the PVD database.

Dear VVV and Ivek,

Meanwhile, I have experienced another strange issue. While updating some movies, it asks me to confirm the movie, although IMDb urls are already in database, and for for twice!. You can check comparing offered url on the first screenshot (question marks), with the url already in database in the second screenshot (red arrow) - they are the same.

Also, for some movies I get infobox "Movie not found " although IMDb url exists - please see third screenshot?!

It didn't happen earlier.

It's like script doesn't use existing urls. Why is that?

Any idea? Thank you in advance.

There is a possibility that duplicates are people for certain reasons, such as different url addresses (or they are not at all) for people. I have already noticed this in my IMDB records for people. For  the IMDB plugin was the download URL of the Imdb reference pages without " www " in the url address. IMDB script has downloaded a full url with Main (overview) Imdb pages.

Finally, I had a similar problem with the TheMovieDB [API] script, where there was a problem with url titles for the Director because the script code was incorrectly written in the script and there were duplicate names for the director because of this, but not for all, only in certain cases.

I mean, and I'm not sure at all, when we started to use IMDb scripts I noticed that I have duplicate entries for people. It's like some people were imported by nostra's imdb.dll and their duplicates with scripts. So now, for example I have two (sometimes, even more) actors with the same name in database and their filmography is split (to movies imported by imdb.dll and other part with script).

Hello,

I just wanted to ask if it's possible FA to have it's own rating custom field?

Thank you in advance

AddCustomFieldValueByName('FilmAffinity_Rating',ItemValue);