Personal Video Database

Attach you have a functional version:
VERSION:    0.3.0.0 (06/11/2016) -> VVV: Rebuild full script.

It retrieve the principal infos (you could complete the rest, I left hints in the code).
I don't now all the custom fields. I retrieved some ratings but I left hints and questions in the code.
Not poster or screenshot (if you have the https link I could retrieve it with curl and charge in PVdB by file but it's not easy to find in the page).

I see that RT is a slowly page, don't worry if the DOS windows stops a 5 or 6 seconds when you run in PVdB, the script waits (this was a big problem. now is solved).

How to edit or connect now all files from curl_try to Rottentomatoes_ [HTTPS] in one executable file.

The script works directly in PVdB without need get out to DOS (curl_try was only a check).

To spread in internet, the best form to save the 4 needed files:
              Scripts/Rottentomatoes_[HTTPS].psf,
              Scripts/curl-ca-bundle.crt,
              Scripts/curl.exe,
              Scripts/libcurl.dll
is compressed in a zip file but the length is out of the limit of the forum. Now, the script checks that all files are present.

If you want to download the original ones: https://bintray.com/artifact/download/vszakats/generic/curl-7.50.2-win32-mingw.7z (The needed files were in "curl-7.50.2-win32-mingw\bin\" folder)

Will now required for each movie in curl_try file manually enter the URL address of the movie, which is of course more movies lengthy work, or it will be automatically regulated.

Now the script have a automatical full search, but it must save in the 'http' (not S) base url. For back compatibility and for prevents PVdB crash. Do the same if you write it manually.

Finally, it's not difficult to adapt this script to another HTTPS page. All people are free to use it.

Don't worry, all used and created files (downpage.htm) must be in the PVdB Scripts folder.
We must do two tries.
1) Use the new script. It automatically detects the scripts folder were must be the Rottentomatoes_[HTTPS].psf and the three curl files. Send to the forum the Log windows with the http://www.rottentomatoes.com/m/godfather try PVdB movie (don't worry if it don't scrapt the information).
2) Decompress the curl_try.zip in the script folder (ever with the three curl files), run curl_try.bat it in DOS windows and copy the DOS text result to the forum.

When we resolves the curl issue we can begin with the scrap information work.

Well, nobody say it was be easy. We can begin the "debug":
1) ¿Do you have this (Script_Folder.jpg) in your PVdB Scripts Folder?





2) ¿Do you change
      PVdB_SCRIPTS_PATH_FOLDER   = 'C:\Users\Public\Portables\PersonalVideoDB\Scripts\';  //The PVdB scripts path folder
to your own script folder (don't forget the last '\')
3) Try with an movie example with his BASE_URL not httpS (Capture2.jpg)




and Import with Rottentomatoes_[HTTPS].psf in debug mode. ¿can you see in the Log Windows, something like this (LogWindow.jpg)?





If not ¿can you attach your Log window?

¡How can I refuse help you!

I changed a little your Rottentomatoes script to resolve with curl the https issue:
¡First, do a backup of your PVdb folder!
Descompress the attach in your PVdB folder (all goes to scripts folder, I must make two files in to messages because the maximum length it limited in the forum) and change in the script the PVdB_SCRIPTS_PATH_FOLDER.
Launch PVdB in debug mode (debug.bat) and you can see what the script do in the  Help/Log window.

Some explanation of PVdB Import operation:
1) It use the 'GetBaseURL' result to try download the movie page directly. So you must cheat with a false url for avoid download the https page. so ¿you need search the movie always? Don't worry you search the stored movie URL by yourself in 'GetDownloadURL'
2) PVdB don't have a URL so it use 'GetDownloadURL' for download it.
ROUNDABOUT: In this function we download the page (movie, search, etc) with curl and give it a file in the place of a URL.
2.1) If we are in search mode we look for stored movie URL, download the page, change a mode normal and return the file with the page.
2.2) If we don't have URL, we download the search page, continue in search mode and return the file with the page.
2.3) If other modes, you can do similar changing the script mode in order to pass the information to the 'Parsepage' PBdV function.
3) PVdB goes to 'ParsePage' to work over HTML variable. Nothing it's diferent here: HTML have the page info (same behaviour file or web), script mode for knows the type of page information (search, movie, ratting, poster, etc).

Some advices:
1) For editing I use PSPad with 'Object Pascal' highlighted syntax (easier detects some error)
2) I save the html page and I put the script and the page in the editor 'vertical half window mosaic' and it's easy to avance scraping the information.
3) Use, use, use the 'LogMessage' command with the PVdB Help/Log window in debug.bat mode. First you can see the compiling errors. Running, you can see the script flow and the variable contents.
4) The PVdB script is nearly and word process search, find and copy. Don't be afraid of programming. BTW, now I use a lot 'TextBetWeen' function for retrieving info. You can see in my FilmAffinity_[ES].psf:
      //Get ~orating~
      ItemValue:=TextBetWeen(HTML,'<div id="rat-avg-container">','</div>',false,curPos);     //Strings which opens/closes the data. WEB_SPECIFIC
      ItemValue:=StringReplace(ItemValue,',','.',True,True,False); //Decimal comma spanish separator to point english separator.
      AddFieldValueXML('orname',RATING_NAME);
      AddFieldValueXML('orating',ItemValue);
      LogMessage('      Get result orating:'+ItemValue+'||');

BTW You can see the highlighted syntax effect and the use of 'LogMessage'
5) Now, I only use the ParsePage (it obligatory) for scrap and I only have one flow. I diference the script mode with:
      if (Mode=smSearch) then begin       //In search Mode
          ......
          Mode:=smNormal;
          Result:=prList;     //Don't work with Preferences/Plugings/Silent Mode.
          LogMessage('After parsing search Movies go to choose List Results');   
          Exit;
      end;
      if (Mode=smNormal) then begin        //In normal, movie info, Mode
          ......
          Result:=prFinished;
          exit;
     end;

I hope that this help you, I try the 'curl Rottentomatoes script' and it download well the info but I don't have time to make the scrap (the information search of the script). Tell me your avance and problems.
(Rename the attach as Script.001.zip -> Scripts.zip.001)

I think that we have PVdB (PDV) for long time if Ivek keep the communication alive. Thank you very much. (BTW I wish to Nostra best with his health, too).

I take the opportunity to tell my story for explain the first affirmation.

• Year 2003) My first player was a Kiss DP-500 and my database Excel
• Year 2008) Player 'WD TV HD Media Player Gen1' with a the ModFirmware B-RAD.CC_WDLXTV. It can navigate in the 'background.jpg' stored in the film folder. If you write the film info in the jpg you have a good player. ¿How do it? With TViXiE, it scrap and make the background (BTW, I still use the backgrounds in the film folder, it very useful in USB, even you can see in TV)
  
• Year 2011) XBMC arrives. It scrap and its easy to use, a good media player but it's slow in scrap, you must rebuild the bibliotheque and you can save the film info in the folder but it's difficult of update.
• Year 2013)  To much work for keep update my collection, TViXiE (and the scrapers fails), KODI, etc.  I landed to PVdB and I build my collection. What can i do?
  

   
I make my own scrapers http://www.videodb.info/forum_en/index.php/topic,3876.msg19888.html#msg19888 http://www.videodb.info/forum_en/index.php/topic,3881.msg19414.html#msg19414.

I make an PVdB export script for give the info to TViXiE and with a AutoHotKey script automatically I can rebuild all backgrounds with fresh info http://www.videodb.info/forum_en/index.php/topic,3880.msg18295.html#msg18295.

I make an PVdB export script for give to KODY the .nfo file, poster, fanart and I can refresh all info automatically and KODY only must charge from disc.http://www.videodb.info/forum_en/index.php/topic,1496.msg18294.html#msg18294

And more ... I make an scraper that search the best films (rating and votes), compares with my collection and propose me the good film for not lose my time.

¡Long life for PVdB (PDV)!     

PS: Ivek don't worry about the https issue. There is a roundabout: we can use a external program (I think in curl https://curl.haxx.se/) to download the page to a file and scrap it with PVdB.

Update:

CHANGE LOG (20/10/2016):
            V 1.2.0.0   VVV: Clean code with TextBetWeen (RemoveTags not needed) and ExplodeString functions. Use of TWideArray variable type.
                             Use 'Trim' PDV funtion in place of "CleanBeginEndSpaces" user function. 

Attached the new version.

(Edito: New versions down)

Update:

CHANGE LOG (17/09/2016):
            V 0.2.1.7   VVV: Adjust WEB_SPECIFIC to new API (Problems with search with several results: Choice of order, No poster).
Attached the new version.

(Edito: New versions down)

Sorry  , I had not seen so many emails (I added in the script the link of a contact Web for quick alert)
Thank very much, Ivek23 for maintaining the program  .

I have consolidated the Ivek23 version and fix the search new API

CHANGE LOG (06/06/2016):
            V 0.2.1.6   VVV: Adjust WEB_SPECIFIC to new API (Problems with search with several results).
            V 0.2.1.5   Ivek23: Adjust WEB_SPECIFIC to new API (Problems with composer).
Attached the new version.

FilmAffinity_New_Releases.psf
Need: Search in FilmAffinity DVD rental films the movies with a rating higher of 6 and number of votes greater than 2000.
Problem to solve: Select from a web list and then gets information (for instance, number of votes) of those movies.
Characteristics of Script: A script of multiple passes. Search the list page (stored in the first access) and while the list is parsed, gets the pages from the good movies to get more information (searches similar to "FilmAffinity_[ES]"). All is saved to a file. If GET_ONLY_NOT_PVdB = True, gets only the movies not present in PVdB (but PVdb_CSVDB_FILE is needed). Although not very smart, use global variables for better understanding.

Attached the new version too.

(Edito: New versions down)

CHANGE LOG (08/01/2016):
            V 0.2.0.5   VVV: Fixed bug with the "total_results" detection (before it was the last information now "total_pages" is the last).
                             Adjust to the new order of received informations for "Parsing search Movies"
                             Eliminate "2014" from name.

Attached the new versión.

(Thanks again Ivek23)

(Edito: New version down)

CHANGE LOG (22/11/2015):
            V 0.2.1.3   VVV: Writes 'ratingValue'=orating with point english separator for better standarisation.
                             Eliminate "2014" from name.
Attached the new versión.

(Edito: New version down)