Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - VVV_Easy_Programing

Pages: 1 2 3 4 5 6 [7] 8 9 10
121
Talk / Re: curl - solution for https
« on: November 08, 2016, 07:52:31 pm »
I made and attach the next version:
VERSION:    0.3.1.0 (06/11/2016) -> VVV: Use quotation marks for the paths.

but I see some things bizarre in you log file:
Quote
(8.11.2016 8:46:09) PVD Version: 0.9.9.21
¿why you don't use the latest version 1.0.2.7?

Quote
(8.11.2016 8:46:37) UpdateToolbar: 4
(8.11.2016 8:46:45) GET: http://www.videodb.info/upload/check.php
(8.11.2016 8:46:47) Update error: Socket Error # 10061
Connection refused.
(8.11.2016 8:46:52) UpdateToolbar: 5
It seems that your copy of PVdB have problems even with the http pages ¿are you sure that you haven't a proxy, firewall or antivirus with your PVdB? It's very rare that curl works out of PVdB and not in ¿are you sure that Proxomitron is not active?.

¿Can you test it in a clean downloaded 1.0.2.7 version? You can use a portable one decompressing:
http://www.videodb.info/bin/pvd1.zip
in C:\Users\Ivo\Documents\PersonalVideoDB\ and adding the three needed curl files.
Perhaps other people can help us trying the script.

BTW:
The biggest solved of script is synchronize PVdB and the curl download and be sure when you parse that 'downpage.htm' is fresh and completed. So I do two steeps:
1) First the script delete the saved 'downpage.htm'.
Quote
    //Delete the ancient dowloaded page file. Needed for wait to curl download.       
    While FileExists(ScriptPath+BASE_DOWNLOAD_FILE) do begin
         FileExecute('cmd.exe', '/C del '+ScriptPath+BASE_DOWNLOAD_FILE);
         LogMessage('      Waiting 1s for delete:'+ScriptPath+BASE_DOWNLOAD_FILE);
         wait (1000);
    end;
2) After curl call it waits to detected the  'downpage.htm'
Quote
    //Wait curl finish and exist the downloaded page
    While Not(FileExists(ScriptPath+BASE_DOWNLOAD_FILE)) do begin
         LogMessage('      Waiting 2s for exists of:'+ScriptPath+BASE_DOWNLOAD_FILE);
         wait (2000);
    end;
So in your first log, the script has problems to delete 'downpage.htm', it tries and wait, tries and wait:
Quote
(8.11.2016 8:47:03)       Waiting 1s for delete:C:\Program Files\Personal Video Database\Scripts\downpage.htm
(8.11.2016 8:47:04)       Waiting 1s for delete:C:\Program Files\Personal Video Database\Scripts\downpage.htm
(8.11.2016 8:47:05)       Waiting 1s for delete:C:\Program Files\Personal Video Database\Scripts\downpage.htm
When you block this part of code it detects that something goes wrong:
Quote
(8.11.2016 8:54:58) Global Var-ScriptPath|C:\Program Files\Personal Video Database\Scripts\|
(8.11.2016 8:54:58)       Some error deleting: C:\Program Files\Personal Video Database\Scripts\downpage.htm


 

122
Talk / Re: curl - solution for https
« on: November 08, 2016, 01:23:32 pm »
I think that the problem it that you program path have spaces and the 'del' command don't be able to erase 'downpage.htm' (it tries, wait 1 seg and tries, tries ...)
Perhaps curl may the same problem.

This afternoon I shall try something with inverted commas but ¿may you do a try with a copy of PVdB in a folder without spaces, for instance: C:\Users\Ivo\Documents\PersonalVideoDB\?

123
Talk / Re: curl - solution for https
« on: November 06, 2016, 08:34:08 pm »
Attach you have a functional version:
VERSION:    0.3.0.0 (06/11/2016) -> VVV: Rebuild full script.

It retrieve the principal infos (you could complete the rest, I left hints in the code).
I don't now all the custom fields. I retrieved some ratings but I left hints and questions in the code.
Not poster or screenshot (if you have the https link I could retrieve it with curl and charge in PVdB by file but it's not easy to find in the page).

I see that RT is a slowly page, don't worry if the DOS windows stops a 5 or 6 seconds when you run in PVdB, the script waits (this was a big problem. now is solved).

Quote
How to edit or connect now all files from curl_try to Rottentomatoes_ [HTTPS] in one executable file.
The script works directly in PVdB without need get out to DOS (curl_try was only a check).

To spread in internet, the best form to save the 4 needed files:
              Scripts/Rottentomatoes_[HTTPS].psf,
              Scripts/curl-ca-bundle.crt,
              Scripts/curl.exe,
              Scripts/libcurl.dll
is compressed in a zip file but the length is out of the limit of the forum. Now, the script checks that all files are present.

If you want to download the original ones: https://bintray.com/artifact/download/vszakats/generic/curl-7.50.2-win32-mingw.7z (The needed files were in "curl-7.50.2-win32-mingw\bin\" folder)

Quote
Will now required for each movie in curl_try file manually enter the URL address of the movie, which is of course more movies lengthy work, or it will be automatically regulated.
Now the script have a automatical full search, but it must save in the 'http' (not S) base url. For back compatibility and for prevents PVdB crash. Do the same if you write it manually.

Finally, it's not difficult to adapt this script to another HTTPS page. All people are free to use it.

124
Talk / Re: curl - solution for https
« on: November 05, 2016, 12:19:57 pm »
Good news.
I must ask you some questions.
¿Do you want make a complete script or only one for your custom fields?
If complete: ¿Do you want that I help you with the script? ¿May I you propose some improvements?
1) I attach: 0.2.0.3 (05/11/2016) -> VVV: Some improvements and explanations in code. without touch the code
2) For touching the code:
    I see that the structure is a little bit confusing ¿do you want that I arrange it?
    I see that the script use AddFieldValue that is deprecated. It better AddFieldValueXML ¿do you want that I arrange it?
    I think that the ParseSearchResults don't work ¿do you want that I programme it?

I don't maintain another script but I can help you.

125
Talk / Re: curl - solution for https
« on: November 04, 2016, 09:38:33 pm »
Don't worry, all used and created files (downpage.htm) must be in the PVdB Scripts folder.
We must do two tries.
1) Use the new script. It automatically detects the scripts folder were must be the Rottentomatoes_[HTTPS].psf and the three curl files. Send to the forum the Log windows with the http://www.rottentomatoes.com/m/godfather try PVdB movie (don't worry if it don't scrapt the information).
2) Decompress the curl_try.zip in the script folder (ever with the three curl files), run curl_try.bat it in DOS windows and copy the DOS text result to the forum.

When we resolves the curl issue we can begin with the scrap information work.

126
Talk / Re: curl - solution for https
« on: November 03, 2016, 08:46:05 pm »
Bad news.
 I don't see an special authorization for curl in my windows firewall.
         My firewall is windows  ¿Do you have a firewall different that windows standard?
         I have Firefox installed ¿Do you have it?
         My antivirus is avast ¿do you have a especial antivirus?
         ¿What it is the content of your downloaded file? ¿Some like this or nothing at all?
                 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
                 <html><head>
                <title>403 Forbidden</title>

¿Can you try this commands in DOS Windows in the script folder?

1)Download a httpS in file not silent
curl.exe -o downpage_try.htm https://www.rottentomatoes.com/m/godfather

I get the page and curl shows
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  222k  100  222k    0     0  87034      0  0:00:02  0:00:02 --:--:-- 87034


2)Download a http in file not silent
curl.exe -o downpage_try.htm http://www.imdb.com/title/tt0068646/?ref_=fn_al_tt_7

I get the page and curl shows
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  197k    0  197k    0     0   214k      0 --:--:-- --:--:-- --:--:--  214k


3)Download a http in file not silent with identificated agent:
curl -A "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)" -o downpage_try.htm http://www.filmaffinity.com/es/film809297.html

I get the page and curl shows
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Curren
                                 Dload  Upload   Total   Spent    Left  Speed
100 48198    0 48198    0     0   336k      0 --:--:-- --:--:-- --:--:--  336k



4)If it fail try (verbose, for know if curl says something):
curl.exe -v -o downpage_try.htm http://www.imdb.com/title/tt0068646/?ref_=fn_al_tt_7



127
Talk / Re: curl - solution for https
« on: November 02, 2016, 07:30:51 pm »
Well, nobody say it was be easy. We can begin the "debug":
1) ¿Do you have this (Script_Folder.jpg) in your PVdB Scripts Folder?





2) ¿Do you change
      PVdB_SCRIPTS_PATH_FOLDER   = 'C:\Users\Public\Portables\PersonalVideoDB\Scripts\'//The PVdB scripts path folder
to your own script folder (don't forget the last '\')
3) Try with an movie example with his BASE_URL not httpS (Capture2.jpg)




and Import with Rottentomatoes_[HTTPS].psf in debug mode. ¿can you see in the Log Windows, something like this (LogWindow.jpg)?





If not ¿can you attach your Log window?

128
Talk / Re: curl - solution for https
« on: November 01, 2016, 09:57:26 pm »

Rename the attach as Script.002.zip -> Scripts.zip.002. Decompress the first with 7zip. It call to the second automatically.


129
Talk / Re: curl - solution for https
« on: November 01, 2016, 09:55:56 pm »
¡How can I refuse help you!

I changed a little your Rottentomatoes script to resolve with curl the https issue:
¡First, do a backup of your PVdb folder!
Descompress the attach in your PVdB folder (all goes to scripts folder, I must make two files in to messages because the maximum length it limited in the forum) and change in the script the PVdB_SCRIPTS_PATH_FOLDER.
Launch PVdB in debug mode (debug.bat) and you can see what the script do in the  Help/Log window.

Some explanation of PVdB Import operation:
1) It use the 'GetBaseURL' result to try download the movie page directly. So you must cheat with a false url for avoid download the https page. so ¿you need search the movie always? Don't worry you search the stored movie URL by yourself in 'GetDownloadURL'
2) PVdB don't have a URL so it use 'GetDownloadURL' for download it.
ROUNDABOUT: In this function we download the page (movie, search, etc) with curl and give it a file in the place of a URL.
2.1) If we are in search mode we look for stored movie URL, download the page, change a mode normal and return the file with the page.
2.2) If we don't have URL, we download the search page, continue in search mode and return the file with the page.
2.3) If other modes, you can do similar changing the script mode in order to pass the information to the 'Parsepage' PBdV function.
3) PVdB goes to 'ParsePage' to work over HTML variable. Nothing it's diferent here: HTML have the page info (same behaviour file or web), script mode for knows the type of page information (search, movie, ratting, poster, etc).

Some advices:
1) For editing I use PSPad with 'Object Pascal' highlighted syntax (easier detects some error)
2) I save the html page and I put the script and the page in the editor 'vertical half window mosaic' and it's easy to avance scraping the information.
3) Use, use, use the 'LogMessage' command with the PVdB Help/Log window in debug.bat mode. First you can see the compiling errors. Running, you can see the script flow and the variable contents.
4) The PVdB script is nearly and word process search, find and copy. Don't be afraid of programming. BTW, now I use a lot 'TextBetWeen' function for retrieving info. You can see in my FilmAffinity_[ES].psf:
      //Get ~orating~
      ItemValue:=TextBetWeen(HTML,'<div id="rat-avg-container">','</div>',false,curPos);     //Strings which opens/closes the data. WEB_SPECIFIC
      ItemValue:=StringReplace(ItemValue,',','.',True,True,False); //Decimal comma spanish separator to point english separator.
      AddFieldValueXML('orname',RATING_NAME);
      AddFieldValueXML('orating',ItemValue);
      LogMessage('      Get result orating:'+ItemValue+'||');

BTW You can see the highlighted syntax effect and the use of 'LogMessage'
5) Now, I only use the ParsePage (it obligatory) for scrap and I only have one flow. I diference the script mode with:
      if (Mode=smSearch) then begin       //In search Mode
          ......
          Mode:=smNormal;
          Result:=prList;     //Don't work with Preferences/Plugings/Silent Mode.
          LogMessage('After parsing search Movies go to choose List Results');   
          Exit;
      end;
      if (Mode=smNormal) then begin        //In normal, movie info, Mode
          ......
          Result:=prFinished;
          exit;
     end;

I hope that this help you, I try the 'curl Rottentomatoes script' and it download well the info but I don't have time to make the scrap (the information search of the script). Tell me your avance and problems.
(Rename the attach as Script.001.zip -> Scripts.zip.001)

130
Talk / Re: curl - solution for https
« on: October 30, 2016, 09:34:53 pm »
You don't need to program out of PVdB. curl.exe is a program that download a web page (even https) to a file in the command line so:

In 'GetDownloadURL' PVdB function:
1) Download the page https (for instance: https://www.themoviedb.org/movie/238-the-godfather) with the next command using FileExecute PVdB procedure:
curl -s -o downpage.htm https://www.themoviedb.org/movie/238-the-godfather
(Now you have the page in the file 'downpage.htm' . Easy to scrap ¿no?)

2) Cheat the 'GET' PVdB funtion as roundabout to the "https" fail with a false URL return BASE_URL_RONDABOUT = 'RONDABOUT'.
(You can get inspiration in my TheMovieDB_[ES].psf script, or, perhaps you can use a file as dummy search string as in my Several_File_Infos.psf)

3) When PVdB return 'empty' to the obligatory callback function ParsePage you can parse the page file with HTML:=FileToString(downpage.htm');

Well, I hope it serve you as inspiration (I explain it fast because I don't have much time).
Needed external:
1) Download the curl-7.50.2-win32-mingw.7z file from https://bintray.com/artifact/download/vszakats/generic/curl-7.50.2-win32-mingw.7z   (Thanks Viktor Szakáts).
2)Extract the three curl libraries files and copy then to script folder:
• curl-7.50.2-win32-mingw\bin\curl.exe
• curl-7.50.2-win32-mingw\bin\curl-ca-bundle.crt
• curl-7.50.2-win32-mingw\bin\libcurl.dll


131
Talk / Re: Videodb.info news
« on: October 29, 2016, 07:00:38 pm »
I think that we have PVdB (PDV) for long time if Ivek keep the communication alive. Thank you very much. (BTW I wish to Nostra best with his health, too).

I take the opportunity to tell my story for explain the first affirmation.
  • Year 2003) My first player was a Kiss DP-500 and my database Excel
  • Year 2008) Player 'WD TV HD Media Player Gen1' with a the ModFirmware B-RAD.CC_WDLXTV. It can navigate in the 'background.jpg' stored in the film folder. If you write the film info in the jpg you have a good player. ¿How do it? With TViXiE, it scrap and make the background (BTW, I still use the backgrounds in the film folder, it very useful in USB, even you can see in TV)
  • Year 2011) XBMC arrives. It scrap and its easy to use, a good media player but it's slow in scrap, you must rebuild the bibliotheque and you can save the film info in the folder but it's difficult of update.
  • Year 2013)  >:( To much work for keep update my collection, TViXiE (and the scrapers fails), KODI, etc.  ;D I landed to PVdB and I build my collection. What can i do?
   
I make my own scrapers http://www.videodb.info/forum_en/index.php/topic,3876.msg19888.html#msg19888 http://www.videodb.info/forum_en/index.php/topic,3881.msg19414.html#msg19414.

I make an PVdB export script for give the info to TViXiE and with a AutoHotKey script automatically I can rebuild all backgrounds with fresh info http://www.videodb.info/forum_en/index.php/topic,3880.msg18295.html#msg18295.

I make an PVdB export script for give to KODY the .nfo file, poster, fanart and I can refresh all info automatically and KODY only must charge from disc.http://www.videodb.info/forum_en/index.php/topic,1496.msg18294.html#msg18294

And more ... I make an scraper that search the best films (rating and votes), compares with my collection and propose me the good film for not lose my time.

¡Long life for PVdB (PDV)!     

PS: Ivek don't worry about the https issue. There is a roundabout: we can use a external program (I think in curl https://curl.haxx.se/) to download the page to a file and scrap it with PVdB.

132
Scripts and Templates / Re: Script: FilmAffinity [ES] (Easy Script)
« on: October 29, 2016, 05:43:29 pm »
Update:

CHANGE LOG (29/10/2016):
            V 1.3.0.0   VVV: Managed redirected film pages in search mode. Added search stored URL (if present) in 'GetDownloadURL' function.
Attached the new version.

Thanks  jippo (jdr___@hotmail.com) for the alert.

(Edito: New versions down)

133
Scripts and Templates / Re: Script: FilmAffinity [ES] (Easy Script)
« on: October 20, 2016, 08:12:43 pm »
Update:

CHANGE LOG (20/10/2016):
            V 1.2.0.0   VVV: Clean code with TextBetWeen (RemoveTags not needed) and ExplodeString functions. Use of TWideArray variable type.
                             Use 'Trim' PDV funtion in place of "CleanBeginEndSpaces" user function. 

Attached the new version.

(Edito: New versions down)

134
Scripts and Templates / Re: Script: FilmAffinity [ES] (Easy Script)
« on: September 25, 2016, 09:40:43 pm »
Update:

CHANGE LOG (25/09/2016):
            V 1.1.0.0   VVV: Use the RemoveTags function for scrap. New "CleanBeginEndSpaces" user function.
 Attached the new version.

Thanks  jippo (jdr___@hotmail.com) by the alert.

(Edito: New versions down)

135
Scripts and Templates / Re: Script: FilmAffinity [ES] (Easy Script)
« on: September 17, 2016, 09:54:17 pm »
Update:

CHANGE LOG (17/09/2016):
            V 0.2.1.7   VVV: Adjust WEB_SPECIFIC to new API (Problems with search with several results: Choice of order, No poster).
Attached the new version.

(Edito: New versions down)

136
Feature Suggestions / Re: Trailers from themoviedb.org
« on: July 06, 2016, 08:20:39 pm »
As noted above there are only two options: directly read the website of the movie (jippo solution) or ask the api of TheMovieDB (Ivek23 solution).
The last may be the most consistent, at the end of the file seem returns the url of the trailer:

"videos":{"results":[{"id":"533ec657c3a368544800078b","iso_639_1":"en","iso_3166_1":"US","key":"R1hZNHPVVAQ","name":"Trailer 1","site":"YouTube","size":360,"type":"Trailer"}]}}

But I don't see this easy and ... ¿how many times see you the trailer of a film that you have seen? (and to decide which movie see never trust in a commercial trailer) ;-)

137
Scripts and Templates / Re: Script: FilmAffinity [ES] (Easy Script)
« on: July 06, 2016, 07:27:09 pm »
Sorry  :'(, I had not seen so many emails (I added in the script the link of a contact Web for quick alert)
Thank very much, Ivek23 for maintaining the program  ;D.

I have consolidated the Ivek23 version and fix the search new API

CHANGE LOG (06/06/2016):
            V 0.2.1.6   VVV: Adjust WEB_SPECIFIC to new API (Problems with search with several results).
            V 0.2.1.5   Ivek23: Adjust WEB_SPECIFIC to new API (Problems with composer).
Attached the new version.

FilmAffinity_New_Releases.psf
Need: Search in FilmAffinity DVD rental films the movies with a rating higher of 6 and number of votes greater than 2000.
Problem to solve: Select from a web list and then gets information (for instance, number of votes) of those movies.
Characteristics of Script: A script of multiple passes. Search the list page (stored in the first access) and while the list is parsed, gets the pages from the good movies to get more information (searches similar to "FilmAffinity_[ES]"). All is saved to a file. If GET_ONLY_NOT_PVdB = True, gets only the movies not present in PVdB (but PVdb_CSVDB_FILE is needed). Although not very smart, use global variables for better understanding.

Attached the new version too.

(Edito: New versions down)

138
Scripts and Templates / Re: Script: FilmAffinity [ES] (Easy Script)
« on: May 22, 2016, 04:55:44 pm »
Sorry  :'(, I had not seen so many emails (I added in the script the link of a contact Web for quick alert)
Thank very much, Ivek23 for maintaining the program  ;D.

I tried to fix all (included the search list) and I fixed the "FilmAffinity_New_Releases.psf" script too.

CHANGE LOG (22/05/2016):
            V 0.2.1.4   VVV: Adjust WEB_SPECIFIC to new API.
                             Contact Web: http://contactbyweb.com/vvv-easy
Attached the new version.
(Edito: New version down)

Thanks Am-rA and jippo, too.

139
Scripts and Templates / Re: Script: TheMovieDB [ES] 2014 (Easy Script)
« on: January 08, 2016, 03:07:57 pm »
CHANGE LOG (08/01/2016):
            V 0.2.0.5   VVV: Fixed bug with the "total_results" detection (before it was the last information now "total_pages" is the last).
                             Adjust to the new order of received informations for "Parsing search Movies"
                             Eliminate "2014" from name.

Attached the new versión.

(Thanks again Ivek23)

(Edito: New version down)

140
New Script in the first message and name change.

Pages: 1 2 3 4 5 6 [7] 8 9 10
anything