Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - VVV_Easy_Programing

Pages: 1 2 3 4 5 6 [7] 8 9 10
121
Talk / Re: Donations to Ivek
« on: January 20, 2017, 10:03:37 pm »
Welcome again, Ivek.
It true that the forum is a little bit empty without you but don't worry ...  we know how to wait.

 :) I wish you the best for your health eyes and if you need rest your eyes don't hesitate and do it. It's the first priority  :)

122
Scripts and Templates / Re: Export to XBMC
« on: January 20, 2017, 09:22:39 pm »
New version that pass artist thumb url to KODI if you have the URL in the translate name (see http://www.videodb.info/forum_en/index.php/topic,4086.0.html#new ).
You have three templates: Only NFO, NFO+fanart.jpg, NFO+folder.jpg (movie poster).

123
Hello,
it's really difficult to find scripts for retrieve information for people.
Well, I make a little one which get the basic info (photo included) in Spanish (but easy to adapt for English or French, see down) from TheMovieDB.
It has an important feature: Store the URL of the person photo in ~transname~ for send to KODI in a Template (see in the forum)..

---------------------------------------------
SCRIPT:     TheMovieDB_[ES]_People
API URL:    http://docs.themoviedb.apiary.io/
CO-AUTHORS: VVV
Contact Web: http://contactbyweb.com/vvv-easy
---------------------------------------------
CHANGE LOG (15/01/2017):
            v 0.1.0.0   VVV: initial release
---------------------------------------------
FIELD USE:
  This is a script designed to be easy to understand and to adapt. For this reason may be missing some fields and
  the programming may not be smart. It's a only "One Pass" Script and get data in API SEQUENTIAL ORDER.
  The Personal Video DataBase field names in comments are enclosed in "~" so they are easier to find in the script.
  The WEB_SPECIFIC commentary points out the specific text for Web fields.
  Use Log PVdB debug Window for looking search errors.
  (Due a PVdB "GET" scripts funtions seems doesn't work with 'https' URL de v.0.2.0.0 works with API web dialog.
   See 'TheMovieDB_[ES]' script for no API (take information direct from web page) scraper.)
  If desired (PHOTO_URL_IN_TRANSNAME=True), it use the Field ~transname~ for storing the URL to the person photo, for send to KODI in a Template.   
KNOWN ISSUES:
  1. For compiling cuestions functions must be defined in the script before use (not in order of execution).
  2. For the search texts for locating the fields in Web page Null '' value it's not valid. You must replaces all
     escaped HTML characters (like &lt;&gt;&amp;&quot;) with normal characters (like <, >,$, ").
  3. It's a only "One Pass" Script: It DOES NOT makes several calls to the main program to scrape information from various pages of
     the movie in the same site. It only do that for searching mode in a movie list results.
  4. It has a Rondabout in BASE_URL (BASE_URL_TRUE,BASE_URL_RONDABOUT) for "cheat" GET PVdB funtion and rondabout the "https" fail.
  5. Get the ~orating~ (Popularity). It's not documented in PVB Scripting Manual and in the script don't work even working in the skin.
  5. In People database, PVdB don't download automaticaly the photo like in Movie database but using ListSearch it do.
     Set PHOTO_DWN_RONDABOUT  = True to activate the the ImageListSearch as RONDABOUT for download Photos. Because there is not choice (because its
     only one photo) normaly it download without asking but if PVdB begin to ask then with PVdB preference/Plugin/Silent Enable would be
     more confortable for large databases.
---------------------------------------------
//User Options
  BYPASS_SILENT        = False; //Set to True to ensure ShowMessage alerts (then bypass Silent PVdB preferences)
  PHOTO_DWN_RONDABOUT  = True;  (*Set to True to activate the the ImageListSearch as RONDABOUT for download Photos.
                                  Because there is not choice (because its only one photo) normaly it download without asking but if PVdB begin to ask then
                                  with PVdB preference/Plugin/Silent Enable would be more confortable for large databases.*)
  PHOTO_URL_IN_TRANSNAME = True; //Set to False to disable to avoid use the Field ~transname~ for storing the URL to the person photo, for send to KODI in a Template.

  LANG_API_STR         = '&language=es';                                  //WEB_SPECIFIC
  //LANG_API_STR       = '&language=en';                                  //WEB_SPECIFIC
  //LANG_API_STR       = '&language=fr';                                  //WEB_SPECIFIC

(Edito: New version down)

124
Development / Re: curl - PVD to https solution
« on: November 30, 2016, 10:07:11 pm »
Well, I don't know very much over proxy but curl has a lot of options over proxy:

Quote
PROXY
 curl supports both HTTP and SOCKS proxy servers, with optional authentication. It does not have special support for FTP proxy servers since there are no standards for those, but it can still be made to work with many of them. You can also use both HTTP and SOCKS proxies to transfer files to and from FTP servers.

You can try to make your own adaptation with the curl proxy parameters (¿--proxy?) listed in:
https://curl.haxx.se/docs/manpage.html

More info in:
https://curl.haxx.se/docs/manual.html

125
Talk / Re: curl - solution for https
« on: November 26, 2016, 11:14:31 am »
VERSION:    0.3.3.0 (23/11/2016) -> VVV: Added Advanced Search with Bing (http search engine) and some code improvements.

Preview poster in search list is disable with // because give me problems

Quote for Ivek23: But with those three movies the RT search work for me. I had must try the bing search with a roundabout (not present in the attach script) so test it in real conditions.

126
Talk / Re: curl - solution for https
« on: November 17, 2016, 08:03:11 pm »
It's a good idea to include a Bing search in the script for advanced search but I had tried and this is a very rare case.
A lot of films are found with the script in normal search ¿can you write me the name of the film that needs advanced search?

I think that must be another problem ¿the title of your films are in english?
In the search the script use 'title' in place of traditional 'origtitle', you can change it in the code:
Quote
end else begin  //The movie URL not exist, search mode needed. Downloand the search page. 
          DownloadURL:=StringReplace(SEARCH_STR_TRUE,'%s',GetFieldValueXML('title'),True,True,False);DownloadURL:=StringReplace(SEARCH_STR_TRUE,'%s',GetFieldValueXML('title'),True,True,False);


In other side, I put a double security in the script for need advanced search:
Quote
        if (0=Pos('Search Results - Rotten Tomatoes', HTML)) or (0<Pos('Sorry, no results found', HTML)) then begin      //WEB_SPECIFIC
          ShowMessage('Not results. Try again or make an advanced search out of the script.'+Chr(13)+'Write the movie URL (http) in the PVdB manually (remember separated by spaces) and run again.','Rotten Tomatoes Search Results');
 
If you have problems with easy films (for instance: godfather) you can delete '(0=Pos('Search Results - Rotten Tomatoes', HTML)) or' in order to try to reduce the false alerts.

I promise you integrate the bing search in the script but I don't have to much time now. I propose you that you finish Rottentomatoes_ [HTTPS] script and I continue the script adding the Bing search.

127
Scripts and Templates / Re: Script: FilmAffinity [ES] (Easy Script)
« on: November 13, 2016, 07:29:14 pm »
Update:

CHANGE LOG :
            V 1.3.1.1 (12/10/2016) VVV: Added clean ~Writers~ of 'cómic:' and 'libro:'. Separated the '(' ')' elimination for get all cases.

Attached the new version.


(Edito: New versions down)

128
Scripts and Templates / Re: Script: FilmAffinity [ES] (Easy Script)
« on: November 12, 2016, 07:03:53 pm »
Update:

V 1.3.1.0 (12/10/2016) VVV: Clean ~Director~ '(creator)', ~Writers~ '(artículo:)''(historia:)''(libros:)''(novela:)''(novelas:)''(obra:)' and ~Composers~ '(canción:)''(canciones:)''(tema:)''(aka)'

Attached the new version.

(Edito: New versions down)

129
Talk / Re: curl - solution for https
« on: November 09, 2016, 04:48:31 pm »
 :D ;D :D ;D :D ;D

If you arrive, in this script or in other, to need download pictures from a HTTPS link (so binary download), tell me because I think that it will be necessary make some code.

130
Talk / Re: curl - solution for https
« on: November 08, 2016, 07:52:31 pm »
I made and attach the next version:
VERSION:    0.3.1.0 (06/11/2016) -> VVV: Use quotation marks for the paths.

but I see some things bizarre in you log file:
Quote
(8.11.2016 8:46:09) PVD Version: 0.9.9.21
¿why you don't use the latest version 1.0.2.7?

Quote
(8.11.2016 8:46:37) UpdateToolbar: 4
(8.11.2016 8:46:45) GET: http://www.videodb.info/upload/check.php
(8.11.2016 8:46:47) Update error: Socket Error # 10061
Connection refused.
(8.11.2016 8:46:52) UpdateToolbar: 5
It seems that your copy of PVdB have problems even with the http pages ¿are you sure that you haven't a proxy, firewall or antivirus with your PVdB? It's very rare that curl works out of PVdB and not in ¿are you sure that Proxomitron is not active?.

¿Can you test it in a clean downloaded 1.0.2.7 version? You can use a portable one decompressing:
http://www.videodb.info/bin/pvd1.zip
in C:\Users\Ivo\Documents\PersonalVideoDB\ and adding the three needed curl files.
Perhaps other people can help us trying the script.

BTW:
The biggest solved of script is synchronize PVdB and the curl download and be sure when you parse that 'downpage.htm' is fresh and completed. So I do two steeps:
1) First the script delete the saved 'downpage.htm'.
Quote
    //Delete the ancient dowloaded page file. Needed for wait to curl download.       
    While FileExists(ScriptPath+BASE_DOWNLOAD_FILE) do begin
         FileExecute('cmd.exe', '/C del '+ScriptPath+BASE_DOWNLOAD_FILE);
         LogMessage('      Waiting 1s for delete:'+ScriptPath+BASE_DOWNLOAD_FILE);
         wait (1000);
    end;
2) After curl call it waits to detected the  'downpage.htm'
Quote
    //Wait curl finish and exist the downloaded page
    While Not(FileExists(ScriptPath+BASE_DOWNLOAD_FILE)) do begin
         LogMessage('      Waiting 2s for exists of:'+ScriptPath+BASE_DOWNLOAD_FILE);
         wait (2000);
    end;
So in your first log, the script has problems to delete 'downpage.htm', it tries and wait, tries and wait:
Quote
(8.11.2016 8:47:03)       Waiting 1s for delete:C:\Program Files\Personal Video Database\Scripts\downpage.htm
(8.11.2016 8:47:04)       Waiting 1s for delete:C:\Program Files\Personal Video Database\Scripts\downpage.htm
(8.11.2016 8:47:05)       Waiting 1s for delete:C:\Program Files\Personal Video Database\Scripts\downpage.htm
When you block this part of code it detects that something goes wrong:
Quote
(8.11.2016 8:54:58) Global Var-ScriptPath|C:\Program Files\Personal Video Database\Scripts\|
(8.11.2016 8:54:58)       Some error deleting: C:\Program Files\Personal Video Database\Scripts\downpage.htm


 

131
Talk / Re: curl - solution for https
« on: November 08, 2016, 01:23:32 pm »
I think that the problem it that you program path have spaces and the 'del' command don't be able to erase 'downpage.htm' (it tries, wait 1 seg and tries, tries ...)
Perhaps curl may the same problem.

This afternoon I shall try something with inverted commas but ¿may you do a try with a copy of PVdB in a folder without spaces, for instance: C:\Users\Ivo\Documents\PersonalVideoDB\?

132
Talk / Re: curl - solution for https
« on: November 06, 2016, 08:34:08 pm »
Attach you have a functional version:
VERSION:    0.3.0.0 (06/11/2016) -> VVV: Rebuild full script.

It retrieve the principal infos (you could complete the rest, I left hints in the code).
I don't now all the custom fields. I retrieved some ratings but I left hints and questions in the code.
Not poster or screenshot (if you have the https link I could retrieve it with curl and charge in PVdB by file but it's not easy to find in the page).

I see that RT is a slowly page, don't worry if the DOS windows stops a 5 or 6 seconds when you run in PVdB, the script waits (this was a big problem. now is solved).

Quote
How to edit or connect now all files from curl_try to Rottentomatoes_ [HTTPS] in one executable file.
The script works directly in PVdB without need get out to DOS (curl_try was only a check).

To spread in internet, the best form to save the 4 needed files:
              Scripts/Rottentomatoes_[HTTPS].psf,
              Scripts/curl-ca-bundle.crt,
              Scripts/curl.exe,
              Scripts/libcurl.dll
is compressed in a zip file but the length is out of the limit of the forum. Now, the script checks that all files are present.

If you want to download the original ones: https://bintray.com/artifact/download/vszakats/generic/curl-7.50.2-win32-mingw.7z (The needed files were in "curl-7.50.2-win32-mingw\bin\" folder)

Quote
Will now required for each movie in curl_try file manually enter the URL address of the movie, which is of course more movies lengthy work, or it will be automatically regulated.
Now the script have a automatical full search, but it must save in the 'http' (not S) base url. For back compatibility and for prevents PVdB crash. Do the same if you write it manually.

Finally, it's not difficult to adapt this script to another HTTPS page. All people are free to use it.

133
Talk / Re: curl - solution for https
« on: November 05, 2016, 12:19:57 pm »
Good news.
I must ask you some questions.
¿Do you want make a complete script or only one for your custom fields?
If complete: ¿Do you want that I help you with the script? ¿May I you propose some improvements?
1) I attach: 0.2.0.3 (05/11/2016) -> VVV: Some improvements and explanations in code. without touch the code
2) For touching the code:
    I see that the structure is a little bit confusing ¿do you want that I arrange it?
    I see that the script use AddFieldValue that is deprecated. It better AddFieldValueXML ¿do you want that I arrange it?
    I think that the ParseSearchResults don't work ¿do you want that I programme it?

I don't maintain another script but I can help you.

134
Talk / Re: curl - solution for https
« on: November 04, 2016, 09:38:33 pm »
Don't worry, all used and created files (downpage.htm) must be in the PVdB Scripts folder.
We must do two tries.
1) Use the new script. It automatically detects the scripts folder were must be the Rottentomatoes_[HTTPS].psf and the three curl files. Send to the forum the Log windows with the http://www.rottentomatoes.com/m/godfather try PVdB movie (don't worry if it don't scrapt the information).
2) Decompress the curl_try.zip in the script folder (ever with the three curl files), run curl_try.bat it in DOS windows and copy the DOS text result to the forum.

When we resolves the curl issue we can begin with the scrap information work.

135
Talk / Re: curl - solution for https
« on: November 03, 2016, 08:46:05 pm »
Bad news.
 I don't see an special authorization for curl in my windows firewall.
         My firewall is windows  ¿Do you have a firewall different that windows standard?
         I have Firefox installed ¿Do you have it?
         My antivirus is avast ¿do you have a especial antivirus?
         ¿What it is the content of your downloaded file? ¿Some like this or nothing at all?
                 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
                 <html><head>
                <title>403 Forbidden</title>

¿Can you try this commands in DOS Windows in the script folder?

1)Download a httpS in file not silent
curl.exe -o downpage_try.htm https://www.rottentomatoes.com/m/godfather

I get the page and curl shows
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  222k  100  222k    0     0  87034      0  0:00:02  0:00:02 --:--:-- 87034


2)Download a http in file not silent
curl.exe -o downpage_try.htm http://www.imdb.com/title/tt0068646/?ref_=fn_al_tt_7

I get the page and curl shows
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  197k    0  197k    0     0   214k      0 --:--:-- --:--:-- --:--:--  214k


3)Download a http in file not silent with identificated agent:
curl -A "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)" -o downpage_try.htm http://www.filmaffinity.com/es/film809297.html

I get the page and curl shows
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Curren
                                 Dload  Upload   Total   Spent    Left  Speed
100 48198    0 48198    0     0   336k      0 --:--:-- --:--:-- --:--:--  336k



4)If it fail try (verbose, for know if curl says something):
curl.exe -v -o downpage_try.htm http://www.imdb.com/title/tt0068646/?ref_=fn_al_tt_7



136
Talk / Re: curl - solution for https
« on: November 02, 2016, 07:30:51 pm »
Well, nobody say it was be easy. We can begin the "debug":
1) ¿Do you have this (Script_Folder.jpg) in your PVdB Scripts Folder?





2) ¿Do you change
      PVdB_SCRIPTS_PATH_FOLDER   = 'C:\Users\Public\Portables\PersonalVideoDB\Scripts\'//The PVdB scripts path folder
to your own script folder (don't forget the last '\')
3) Try with an movie example with his BASE_URL not httpS (Capture2.jpg)




and Import with Rottentomatoes_[HTTPS].psf in debug mode. ¿can you see in the Log Windows, something like this (LogWindow.jpg)?





If not ¿can you attach your Log window?

137
Talk / Re: curl - solution for https
« on: November 01, 2016, 09:57:26 pm »

Rename the attach as Script.002.zip -> Scripts.zip.002. Decompress the first with 7zip. It call to the second automatically.


138
Talk / Re: curl - solution for https
« on: November 01, 2016, 09:55:56 pm »
¡How can I refuse help you!

I changed a little your Rottentomatoes script to resolve with curl the https issue:
¡First, do a backup of your PVdb folder!
Descompress the attach in your PVdB folder (all goes to scripts folder, I must make two files in to messages because the maximum length it limited in the forum) and change in the script the PVdB_SCRIPTS_PATH_FOLDER.
Launch PVdB in debug mode (debug.bat) and you can see what the script do in the  Help/Log window.

Some explanation of PVdB Import operation:
1) It use the 'GetBaseURL' result to try download the movie page directly. So you must cheat with a false url for avoid download the https page. so ¿you need search the movie always? Don't worry you search the stored movie URL by yourself in 'GetDownloadURL'
2) PVdB don't have a URL so it use 'GetDownloadURL' for download it.
ROUNDABOUT: In this function we download the page (movie, search, etc) with curl and give it a file in the place of a URL.
2.1) If we are in search mode we look for stored movie URL, download the page, change a mode normal and return the file with the page.
2.2) If we don't have URL, we download the search page, continue in search mode and return the file with the page.
2.3) If other modes, you can do similar changing the script mode in order to pass the information to the 'Parsepage' PBdV function.
3) PVdB goes to 'ParsePage' to work over HTML variable. Nothing it's diferent here: HTML have the page info (same behaviour file or web), script mode for knows the type of page information (search, movie, ratting, poster, etc).

Some advices:
1) For editing I use PSPad with 'Object Pascal' highlighted syntax (easier detects some error)
2) I save the html page and I put the script and the page in the editor 'vertical half window mosaic' and it's easy to avance scraping the information.
3) Use, use, use the 'LogMessage' command with the PVdB Help/Log window in debug.bat mode. First you can see the compiling errors. Running, you can see the script flow and the variable contents.
4) The PVdB script is nearly and word process search, find and copy. Don't be afraid of programming. BTW, now I use a lot 'TextBetWeen' function for retrieving info. You can see in my FilmAffinity_[ES].psf:
      //Get ~orating~
      ItemValue:=TextBetWeen(HTML,'<div id="rat-avg-container">','</div>',false,curPos);     //Strings which opens/closes the data. WEB_SPECIFIC
      ItemValue:=StringReplace(ItemValue,',','.',True,True,False); //Decimal comma spanish separator to point english separator.
      AddFieldValueXML('orname',RATING_NAME);
      AddFieldValueXML('orating',ItemValue);
      LogMessage('      Get result orating:'+ItemValue+'||');

BTW You can see the highlighted syntax effect and the use of 'LogMessage'
5) Now, I only use the ParsePage (it obligatory) for scrap and I only have one flow. I diference the script mode with:
      if (Mode=smSearch) then begin       //In search Mode
          ......
          Mode:=smNormal;
          Result:=prList;     //Don't work with Preferences/Plugings/Silent Mode.
          LogMessage('After parsing search Movies go to choose List Results');   
          Exit;
      end;
      if (Mode=smNormal) then begin        //In normal, movie info, Mode
          ......
          Result:=prFinished;
          exit;
     end;

I hope that this help you, I try the 'curl Rottentomatoes script' and it download well the info but I don't have time to make the scrap (the information search of the script). Tell me your avance and problems.
(Rename the attach as Script.001.zip -> Scripts.zip.001)

139
Talk / Re: curl - solution for https
« on: October 30, 2016, 09:34:53 pm »
You don't need to program out of PVdB. curl.exe is a program that download a web page (even https) to a file in the command line so:

In 'GetDownloadURL' PVdB function:
1) Download the page https (for instance: https://www.themoviedb.org/movie/238-the-godfather) with the next command using FileExecute PVdB procedure:
curl -s -o downpage.htm https://www.themoviedb.org/movie/238-the-godfather
(Now you have the page in the file 'downpage.htm' . Easy to scrap ¿no?)

2) Cheat the 'GET' PVdB funtion as roundabout to the "https" fail with a false URL return BASE_URL_RONDABOUT = 'RONDABOUT'.
(You can get inspiration in my TheMovieDB_[ES].psf script, or, perhaps you can use a file as dummy search string as in my Several_File_Infos.psf)

3) When PVdB return 'empty' to the obligatory callback function ParsePage you can parse the page file with HTML:=FileToString(downpage.htm');

Well, I hope it serve you as inspiration (I explain it fast because I don't have much time).
Needed external:
1) Download the curl-7.50.2-win32-mingw.7z file from https://bintray.com/artifact/download/vszakats/generic/curl-7.50.2-win32-mingw.7z   (Thanks Viktor Szakáts).
2)Extract the three curl libraries files and copy then to script folder:
• curl-7.50.2-win32-mingw\bin\curl.exe
• curl-7.50.2-win32-mingw\bin\curl-ca-bundle.crt
• curl-7.50.2-win32-mingw\bin\libcurl.dll


140
Talk / Re: Videodb.info news
« on: October 29, 2016, 07:00:38 pm »
I think that we have PVdB (PDV) for long time if Ivek keep the communication alive. Thank you very much. (BTW I wish to Nostra best with his health, too).

I take the opportunity to tell my story for explain the first affirmation.
  • Year 2003) My first player was a Kiss DP-500 and my database Excel
  • Year 2008) Player 'WD TV HD Media Player Gen1' with a the ModFirmware B-RAD.CC_WDLXTV. It can navigate in the 'background.jpg' stored in the film folder. If you write the film info in the jpg you have a good player. ¿How do it? With TViXiE, it scrap and make the background (BTW, I still use the backgrounds in the film folder, it very useful in USB, even you can see in TV)
  • Year 2011) XBMC arrives. It scrap and its easy to use, a good media player but it's slow in scrap, you must rebuild the bibliotheque and you can save the film info in the folder but it's difficult of update.
  • Year 2013)  >:( To much work for keep update my collection, TViXiE (and the scrapers fails), KODI, etc.  ;D I landed to PVdB and I build my collection. What can i do?
   
I make my own scrapers http://www.videodb.info/forum_en/index.php/topic,3876.msg19888.html#msg19888 http://www.videodb.info/forum_en/index.php/topic,3881.msg19414.html#msg19414.

I make an PVdB export script for give the info to TViXiE and with a AutoHotKey script automatically I can rebuild all backgrounds with fresh info http://www.videodb.info/forum_en/index.php/topic,3880.msg18295.html#msg18295.

I make an PVdB export script for give to KODY the .nfo file, poster, fanart and I can refresh all info automatically and KODY only must charge from disc.http://www.videodb.info/forum_en/index.php/topic,1496.msg18294.html#msg18294

And more ... I make an scraper that search the best films (rating and votes), compares with my collection and propose me the good film for not lose my time.

¡Long life for PVdB (PDV)!     

PS: Ivek don't worry about the https issue. There is a roundabout: we can use a external program (I think in curl https://curl.haxx.se/) to download the page to a file and scrap it with PVdB.

Pages: 1 2 3 4 5 6 [7] 8 9 10