Personal Video Database

English => Support => Topic started by: rick.ca on July 21, 2011, 10:22:25 pm

Title: AllRovi movie script
Post by: rick.ca on July 21, 2011, 10:22:25 pm
Notice:

AllRovi script does not work anymore.




It's topic not very visible on the Scripts and Templates (http://www.videodb.info/forum_en/index.php/board,9.0.html) board, so I'm announcing it here. Thanks to Ivek, we once again have access to most Allmovie data.

For those not familiar with Rovi/Allmovie, its consistent quality of Synopsis and Reviews is much better than the user-provided crap on IMDb.

Now for all former AllMovie fans here is new AllRovi script, based on AllMovie script and with the help of other scripts, especially the new Italian script. The script gets most of the movie information currently available at the new AllRovi site (http://www.allrovi.com/movies/), including:

  • Basic info; Title, Year, Director, Genre, Run time, Release, etc.
  • Poster (not good quality, but sufficient for a thumbnail)
  • Rovi rating
  • Synopsis
  • Review
  • Sub-Genres and Themes
  • MPAA rating and Flags

We (co-authors Ivek23 and rick.ca) have a faint hope more data that was available at the old AllMovie site will be added to the new AllRovi site. If that happens, we'll update the script to include it.

The most current version of the script is always attached to this message. Details of changes are documented in the script, and may be commented on in messages posted to this topic. Please post any comments or questions concerning this script to this topic.
Title: Re: AllRovi movie script
Post by: rick.ca on August 02, 2011, 11:54:17 pm
The script is now very near completion. A "Release Candidate," if you like. I've added a list of it's unique features to it's description...

...and adds some special features:

  • Full Cast and Crew listings with links to Rovi bios.
  • Support for separate Attributes, Themes and Moods, even though Rovi now lumps all these together in Characteristics.
  • Mood links to old AllMovie mood pages (description of the mood and a list of "major works") found in the Web Archive (http://wayback.archive.org/web/).
  • Links to Rovi's Genre and Sub-Genre pages (with description, significant works and 'most visited' films).
Title: Re: AllRovi movie script
Post by: rick.ca on August 04, 2011, 08:06:12 am
The script is now very near completion. A "Release Candidate," if you like.

Who an I kidding? This is not Microsoft. The script is complete and has been released to the auto-update system. If there you have any problems or suggestions for improvement, please post them here.
Title: Re: AllRovi movie script
Post by: minolotus on August 05, 2011, 07:23:49 pm
Even the script is now officially released, I would like to add two comments on the latest version.

I think the new update field is a really good idea (it would be nice if a similar field for the imdb script exists ... but that’s another point). Unfortunately, the date format is not similar to the date format of the "added" and "modified" fields. Would it be possible that the fields use the local settings of the respective computer? 
 
I have another issue with the "modified" field. I am not sure if this is related to the script or a more general problem of PVD. I rerun the script to get the synopsis to a custom field. The script put the data to the field but the "modified" field wasn’t updated.  I checked this with the "characteristics" and the "flags" field (removing the data and rerunning the script). In both cases the "modified" field wasn’t updated?
Title: Re: AllRovi movie script
Post by: Ivek23 on August 05, 2011, 07:52:23 pm
Quote
I think the new update field is a really good idea (it would be nice if a similar field for the imdb script exists ... but that’s another point). Unfortunately, the date format is not similar to the date format of the "added" and "modified" fields.  

"Added" field does not change, remaining from the date of entry when the entry was added, while in the "Modified" field is changing the date in it with every change made ​​to any of the saved in PVD database.

Quote
I have another issue with the "modified" field. I am not sure if this is related to the script or a more general problem of PVD.
I think, this is more
Quote
a more general problem of PVD

What about the custom "Updated" field, there are certain data change.
See attached pictures.


[attachment deleted by admin]
Title: Re: AllRovi movie script
Post by: minolotus on August 05, 2011, 08:09:31 pm
"Added" field does not change, remaining from the date of entry when the entry was added, while in the "Modified" field is changing the date in it with every change made to any of the saved in PVD database.

My point was that the format of the date of the "update" field on the one hand and the "modified" and "added" fields on the other hand are not the same due to the local date settings on my computer. The structure of the update field is "YYYY-MM-DD" and the structure of the other fields mentioned are "DD.MM.YYYY".

Quote
What about the custom "Updated" field, there are certain data change.
See attached pictures.

That's true. The "update" field works fine. But it is important for me that the "modified" filed is updated every time when changes on the data are proceeded.
Title: Re: AllRovi movie script
Post by: rick.ca on August 05, 2011, 08:42:28 pm
Unfortunately, the date format is not similar to the date format of the "added" and "modified" fields. Would it be possible that the fields use the local settings of the respective computer?

The script engine includes the necessary date-handling functions, but it's not possible to add a custom date-type field to version 0.9. It is in version 1.0, so we can make that change when it's released.

Quote
I have another issue with the "modified" field. I am not sure if this is related to the script or a more general problem of PVD. I rerun the script to get the synopsis to a custom field. The script put the data to the field but the "modified" field wasn’t updated.  I checked this with the "characteristics" and the "flags" field (removing the data and rerunning the script). In both cases the "modified" field wasn’t updated?

This is curious. If I change any fields (e.g., removing data to see what happens when the script is rerun), Modified is updated (i.e., immediately—before the script is run). I don't know why that might be different for you. In any case, the main program handles the date fields. Whatever is happening has nothing to do with the script.

Quote
But it is important for me that the "modified" filed is updated every time when changes on the data are proceeded.

As well you should be, if using PvdImport to update J. River Media Center. It relies on an accurate DateTime to determine which records have been updated by PVD. I haven't noticed any problems with records failing to update. Have you?
Title: Re: AllRovi movie script
Post by: minolotus on August 05, 2011, 09:16:52 pm
The script engine includes the necessary date-handling functions, but it's not possible to add a custom date-type field to version 0.9. It is in version 1.0, so we can make that change when it's released.
Thanks, that's great. Looking forward to it.

Quote
This is curious. If I change any fields (e.g., removing data to see what happens when the script is rerun), Modified is updated (i.e., immediately—before the script is run). I don't know why that might be different for you. In any case, the main program handles the date fields. Whatever is happening has nothing to do with the script.

Are you sure that the update of the field is caused by the rerun of the script and not by removing the data before? I check this with the imdb script and I think that the timestamp is set after the update?


Quote
As well you should be, if using PvdImport to update J. River Media Center. It relies on an accurate DateTime to determine which records have been updated by PVD. I haven't noticed any problems with records failing to update. Have you?
Yep, when the "modified" field is not updated, PvDImport does not recognise the update and doen't import the new data.   Then, I need to start the update manually :(

Update: I have just remembered that I had similar problems with the multiple movies editor:

I have discovered that the field “last modified” is not always updated when using the multiple movies editor. This appears only from time to time. Maybe the problem is related to the field to be modified. When I change the genre, the timestamp does not change but after editing the director field of the same movies the “last modified” field is updated.
Title: Re: AllRovi movie script
Post by: rick.ca on August 06, 2011, 12:45:57 am
Quote
Are you sure that the update of the field is caused by the rerun of the script and not by removing the data before? I check this with the imdb script and I think that the timestamp is set after the update?

Modified should be updated whenever any data changes for any reason. I've never had reason to care, but it actually improperly updates if a record is put in edit mode and no changes are made.

Although I haven't tested all fields, I haven't noticed or found any that don't cause Modified to be updated when they are changed by directly editing the record. Have you? Which fields?

Although I haven't tested all plugins and scripts, I've never noticed Modified not being updated when one is run.

Quote
I have just remembered that I had similar problems with the multiple movies editor:

I can reproduce this. I haven't attempted to determine whether fields other than Genre are affected, or whether or not it happens consistently. From what I'm seeing, my guess is the problem is restricted to the MME.

Quote
Yep, when the "modified" field is not updated, PvDImport does not recognise the update and doen't import the new data.

This observation doesn't shed any light on the matter. If Modified has not been updated by PVD, PvdImport is not supposed to re-import the data.

Quote
Then, I need to start the update manually

What do you mean by this? If Modified hasn't changed, you would have to force an update by using the "Send to" menu or run another "Mass import."
Title: Re: AllRovi movie script
Post by: minolotus on August 06, 2011, 01:09:16 pm
I did some tests to narrow down the source of the problem but this wasn’t really successful. I couldn’t discover a pattern in the behaviour of the “modified” field. Here are some of my observations:

-   The problem is not related to a specific script or to the import by scripts in general. The problem could occur with the imdb script and allrovi script, the MME and also in the manual edit mode.
-   I tested also with a new and clean database therefore this should not be problem with a corrupted database. Further I switched between local access to the database and connection to the firebird server but this did have any affects on the problem.
-   My movie skin could be blamed as I tried this also with a standard skin of PVD.
-   There are some fields which causes never any problem, i.e. Budget, box office. When updating the data with a script, deleting or editing the entries manually the “modified” field is updated.


The problem seems to be connected with specific movies as well as with specific fields. Please find attached a table with my testing for four movies. There are some actions which causes always problems (update sub-genre and characteristics with allrovi script). Some other problems occur only for movie 2 and 4.  ???


[attachment deleted by admin]
Title: Re: AllRovi movie script
Post by: minolotus on August 06, 2011, 01:17:59 pm
Quote
Then, I need to start the update manually

What do you mean by this? If Modified hasn't changed, you would have to force an update by using the "Send to" menu or run another "Mass import."

That's what I meant.  :)
Title: Re: AllRovi movie script
Post by: minolotus on August 06, 2011, 06:52:33 pm
-   The problem is not related to a specific script or to the import by scripts in general. The problem could occur with the imdb script and allrovi script, the MME and also in the manual edit mode.

One additional remark regarding the imdb script: In the table in my post above the update with the imdb script always results in an update of the "modified" field. But there are also examples when this is not working: When I remove itemes from the "producer" field. the "modified" filed is always updated. But when I rerun the imdb script the "producer" field is filled properly but the "mofified" filed leaves untouched. I checked this on several movies and got on all of them the same problem.
Title: Re: AllRovi movie script
Post by: rick.ca on August 06, 2011, 08:33:20 pm
Have you included the custom [Updated] field in your database? I have, and it, of course, always changes when the AllRovi script is run. This, in turn, causes [moddate] to be updated. I have no idea why it's not updating as expected, but if running either the plugin or the script will always update it, where does a problem remain? Are there any fields not updated from external sources that do not cause [moddate] to be updated when changed?

The issue I'm seeing, BTW, seems to be sort of the opposite problem. My [moddate] seems to be updated just by putting a record into edit mode—without changing anything. This means, of course, I'm not able to replicate anything you're reporting.
Title: Re: AllRovi movie script
Post by: minolotus on August 06, 2011, 09:30:24 pm
Have you included the custom [Updated] field in your database?
Yes, I have. And this field never causes any problems. It is always updated when running the allrovi script.

Quote
Are there any fields not updated from external sources that do not cause [moddate] to be updated when changed?
I am sorry. Could you please restate your question for me? I have a problem to understand what you mean.

Quote
The issue I'm seeing, BTW, seems to be sort of the opposite problem. My [moddate] seems to be updated just by putting a record into edit mode—without changing anything.
Yes, I discovered this, too, but only for some movies and not for all. At the "Living Daylights, The" for example, the [moddate] field is not updated when entering the edit mode.  

I am stumped  ???

Update: Now, I am getting officially crazy. This morning, when removing the entries from the sub-genre field for “licence to kill” in the edit mode, the [moddate] field was properly updated. But after rerunning the allrovi script, the [moddate] was not updated. I repeated these steps several times and checked carefully the [moddate] field.  So, I am sure that this wasn’t an error in observation.
 
Now, I did this test again and now it is working properly. The [moddate] field is updated in both cases: after removing the content from the field in the edit mode and after running the script.

After this I created a new database and added as custom fields [updated] and [sub-genre]. I added the “licence to kill” movie and ran the imdb and allrovi script. Then, I removed the entries from the sub-genre field and closed the edit mode… The [moddate] field hasn’t been updated  ??? The update with the allrovi script worked fine including an update of the [moddate]  field. I repeated the three times and results were always the same.

I don’t see any pattern in this. I think I will throw my computer away and move into the forests.
Title: Re: AllRovi movie script
Post by: RazorHall on August 06, 2011, 10:07:56 pm
I'm seeing a few movies that aren't getting the Characteristics info, such as Phantom Love (http://www.allrovi.com/movies/movie/phantom-love-v184688).
Title: Re: AllRovi movie script
Post by: Ivek23 on August 07, 2011, 06:20:27 am
I'm seeing a few movies that aren't getting the Characteristics info, such as Phantom Love (http://www.allrovi.com/movies/movie/phantom-love-v184688).

Same cases a major part in Bounty Huntress 2 (2002) (http://www.allrovi.com/movies/movie/bounty-huntress-2-v285853).
Synopsis will be postponed until the end ParseMovie As mentioned here (http://www.videodb.info/forum_en/index.php/topic,2487.80.html), follows attempts:

this code
Code: [Select]
//'Synopsis' saved to ~mfDescription~

curPos := PosFrom('<span>', HTML, curPos);
endPos := PosFrom('</span>', HTML, curPos);
TmpStr5 := TextBetween(HTML, '<span>', '</span>', True, curPos);

curPos := PosFrom('<span>by', HTML, curPos);
endPos := PosFrom('</span>', HTML, curPos);
TmpStr3 := TextBetween(HTML, '<span>by ', '</span>', True, curPos)

curPos := PosFrom('<strong>', HTML, curPos);
endPos := PosFrom('</strong>', HTML, curPos);
TmpStr4 := TextBetween(HTML, '<strong>', '</strong>', True, curPos);

curPos := PosFrom('<p>', HTML, curPos);
endPos := PosFrom('</p>', HTML, curPos);
TmpStr := Copy(HTML, curPos, endPos - curPos);
TmpStr1 := StringReplace(TmpStr, ' -- ', '—', True, True, False);
TmpStr2 := StringReplace(TmpStr1, ' --- ', '—', True, True, False);
TmpStr := StringReplace(TmpStr2, '--', '—', True, True, False);
TmpStr1 := StringReplace(TmpStr, #13#13#13#13, #13#10#13#10, True, True, False);
TmpStr2 := RemoveTagsEx(TmpStr1);

if (TmpStr4 = 'synopsis') AND (Pos('©', TmpStr2) = 0) then
AddFieldValue(mfDescription, TmpStr2 + #13 + '—' + TmpStr3)
else
if (TmpStr4 <> 'synopsis') OR (Pos('©', TmpStr2) = 1) then
LogMessage(' Synopsis: Not available');

Copy here as shown in the attached code
Code: [Select]
//~AMG ID~

TmpStr := HTMLValues(HTML,
'<dt>AMG ID</dt>', '</dd>',
'<pre>', '</pre>',
', ', endPos);
AddCustomFieldValueByName('AMG ID', TmpStr);


//'Synopsis' saved to ~mfDescription~

curPos := PosFrom('<span>', HTML, curPos);
endPos := PosFrom('</span>', HTML, curPos);
TmpStr5 := TextBetween(HTML, '<span>', '</span>', True, curPos);

curPos := PosFrom('<span>by', HTML, curPos);
endPos := PosFrom('</span>', HTML, curPos);
TmpStr3 := TextBetween(HTML, '<span>by ', '</span>', True, curPos)

curPos := PosFrom('<strong>', HTML, curPos);
endPos := PosFrom('</strong>', HTML, curPos);
TmpStr4 := TextBetween(HTML, '<strong>', '</strong>', True, curPos);

curPos := PosFrom('<p>', HTML, curPos);
endPos := PosFrom('</p>', HTML, curPos);
TmpStr := Copy(HTML, curPos, endPos - curPos);
TmpStr1 := StringReplace(TmpStr, ' -- ', '—', True, True, False);
TmpStr2 := StringReplace(TmpStr1, ' --- ', '—', True, True, False);
TmpStr := StringReplace(TmpStr2, '--', '—', True, True, False);
TmpStr1 := StringReplace(TmpStr, #13#13#13#13, #13#10#13#10, True, True, False);
TmpStr2 := RemoveTagsEx(TmpStr1);

if (TmpStr4 = 'synopsis') AND (Pos('©', TmpStr2) = 0) then
AddFieldValue(mfDescription, TmpStr2 + #13 + '—' + TmpStr3)
else
if (TmpStr4 <> 'synopsis') OR (Pos('©', TmpStr2) = 1) then
LogMessage(' Synopsis: Not available');

end;


procedure ParseSearchResults(HTML : String);

and this for me this works perfectly, how it works for you now.
Title: Re: AllRovi movie script
Post by: rick.ca on August 08, 2011, 04:46:53 am
Quote
and this for me this works perfectly, how it works for you now.

Thanks, Ivek. Yes, that seems to work fine. It would be nice to understand why, but that's okay. :-\

I'll upload version 1.1 now. The change is referenced to your message above, so there's no need to post a script or explanation anywhere else.
Title: Re: AllRovi movie script
Post by: Ivek23 on August 08, 2011, 05:00:17 am
Quote
I'll upload version 1.1 now. The change is referenced to your message above, so there's no need to post a script or explanation anywhere else.
OK, I understand.
Title: Re: AllRovi movie script
Post by: deazo on August 08, 2011, 06:18:09 pm

 Hi Ivek23 and Rick, I already thanked you for this but hey thank you again!
 
 I have a slight problem. For some reason, Allrovi script does not gather the rating anymore. I have not changed anything to the preferences, the "additional rating" box is checked. I wonder if there was a change in the last version that caused this?
Title: Re: AllRovi movie script
Post by: Ivek23 on August 08, 2011, 06:26:18 pm
When updating data with AllRovi script for the movie  Dakota (1945) (http://www.allrovi.com/movies/movie/dakota-v11977), I could not find this title in the results table, where several movie with the same title and a different year.
The problem is procedure ParseSearchResults in this part of code:
Code: [Select]

...

//if there are more movies with the same or similar title...

curPos := Pos('<div class="results">', HTML);
if curPos < 1 then
Exit;

curPos := PosFrom('<a href="http://www.allrovi.com/movies/', HTML, curPos);
while curPos > 0 do begin
endPos := PosFrom('">', HTML, curPos);
URL := Copy(HTML, curPos + 31, endPos - curPos - 31);
URL := BASE_URL + URL;

curPos := PosFrom('">', HTML, curPos);
endPos := PosFrom('</a>', HTML, curPos);
Title := Copy(HTML, curPos + 2, endPos - curPos - 2);

curPos := PosFrom('<td class="year">', HTML, curPos);
endPos := PosFrom('</td>', HTML, curPos);
Year := Trim(Copy(HTML, curPos + 17, endPos - curPos - 17));

AddSearchResult(Title, '', Year, URL, '');
curPos := PosFrom('" href="/movies/', HTML, curPos);
end;

end;

function NextMode(curMode : Integer) : Integer;


When I am this one snippet of code  curPos := PosFrom('" href="/movies/', HTML, curPos); replaced by tiny fraction of the code curPos := PosFrom('<a href="http://www.allrovi.com/movies/', HTML, curPos);, now in the results table also previously found above the movie title.

Now this piece of code looks like this:
Code: [Select]

...

//if there are more movies with the same or similar title...

curPos := Pos('<div class="results">', HTML);
if curPos < 1 then
Exit;

curPos := PosFrom('<a href="http://www.allrovi.com/movies/', HTML, curPos);
while curPos > 0 do begin
endPos := PosFrom('">', HTML, curPos);
URL := Copy(HTML, curPos + 31, endPos - curPos - 31);
URL := BASE_URL + URL;

curPos := PosFrom('">', HTML, curPos);
endPos := PosFrom('</a>', HTML, curPos);
Title := Copy(HTML, curPos + 2, endPos - curPos - 2);

curPos := PosFrom('<td class="year">', HTML, curPos);
endPos := PosFrom('</td>', HTML, curPos);
Year := Trim(Copy(HTML, curPos + 17, endPos - curPos - 17));

AddSearchResult(Title, '', Year, URL, '');
curPos := PosFrom('<a href="http://www.allrovi.com/movies/', HTML, curPos);
end;

end;

function NextMode(curMode : Integer) : Integer;


As well as images attached as evidence of how the before and after correcting codes.




[attachment deleted by admin]
Title: Re: AllRovi movie script
Post by: Ivek23 on August 08, 2011, 06:47:58 pm

 Hi Ivek23 and Rick, I already thanked you for this but hey thank you again!
 
 I have a slight problem. For some reason, Allrovi script does not gather the rating anymore. I have not changed anything to the preferences, the "additional rating" box is checked. I wonder if there was a change in the last version that caused this?

Firstly, thanks for the compliment.

I just through auto-update system transferred AllRovi script and test and Rovi rating normally works in the custom field as well as an additional rating if I change this setting
Code: [Select]

GET_RATING = False; //Set to True to add 'Rovi rating' as the standard 'Additonal rating' field

as seen above.

Well look what settings you have, there is probably key to solving the problem.

To demonstrate that everything works properly, the attached images.



[attachment deleted by admin]
Title: Re: AllRovi movie script
Post by: rick.ca on August 08, 2011, 10:07:07 pm
When updating data with AllRovi script for the movie  Dakota (1945) (http://www.allrovi.com/movies/movie/dakota-v11977), I could not find this title in the results table, where several movie with the same title and a different year.
The problem is procedure ParseSearchResults in this part of code:

Thanks. I've been wondering why some search result dialogs had only two entries.

I have a slight problem. For some reason, Allrovi script does not gather the rating anymore. I have not changed anything to the preferences, the "additional rating" box is checked. I wonder if there was a change in the last version that caused this?

As Ivek had pointed out. This is a result of the default Option settings. I was overly cautious in setting some of these to 'False'. They're all controlled by Overwrite settings anyway, so I've changed them all to 'True'.

Uploading version 1.2 now.

Warning: The default options now include the updating of Actors, Duration, Poster, Alternate Rating and (Themes in) Categories. Please ensure your Overwrite fields settings are set accordingly!
Title: Re: AllRovi movie script
Post by: Ivek23 on August 09, 2011, 05:41:17 am
When updating data with AllRovi script for the movie  Dakota (1945) (http://www.allrovi.com/movies/movie/dakota-v11977), I could not find this title in the results table, where several movie with the same title and a different year.
The problem is procedure ParseSearchResults in this part of code:

Thanks. I've been wondering why some search result dialogs had only two entries.


Previously, I never came to such a problem with testing if the script is working properly, I have always found the movie title for testing in two entries.
Title: Re: AllRovi movie script
Post by: rick.ca on August 09, 2011, 06:33:20 am
Quote
Previously, I never came to such a problem with testing if the script is working properly, I have always found the movie title for testing in two entries.

That's probably because they're listed by Title (exact matches) then by Year (descending) then Title (partial matches). If you search for Dakota (1945) (http://www.allrovi.com/search/movies/Dakota), you'll see it's third in the list of search results and therefore wasn't shown. Often duplicate titles are just an original movie and it's remake—so either one would be found in a list of two.

Had you not fixed it, I think someone would have noticed before long. ;)
Title: Re: AllRovi movie script
Post by: Ivek23 on August 09, 2011, 07:07:42 am
If you search for Dakota (1945) (http://www.allrovi.com/search/movies/Dakota), you'll see it's third in the list of search results and therefore wasn't shown. Often duplicate titles are just an original movie and it's remake—so either one would be found in a list of two.

Yeah, I noticed this.
Title: Re: AllRovi movie script
Post by: deazo on August 09, 2011, 11:07:12 am
Quote
I just through auto-update system transferred AllRovi script and test and rating Rovi normally works in the custom field as well as an additional rating if I change this setting
Code:

GET_RATING   = False;   //Set to True to add 'Rovi rating' as the standard 'Additonal rating' field

as seen above.

Thank you Ivek23 it now works very well and I even discovered options I was not using. ;D
Title: Re: AllRovi movie script
Post by: Ivek23 on August 09, 2011, 09:58:37 pm
Thank you Ivek23 it now works very well and I even discovered options I was not using. ;D

Welcome
Title: Re: AllRovi movie script
Post by: RazorHall on August 16, 2011, 08:12:34 pm
Still finding the occasional movie where Characteristics aren't being downloaded.  Lost Lake (http://www.allrovi.com/movies/movie/lost-lake-v323279), for example.
Title: Re: AllRovi movie script
Post by: Ivek23 on August 16, 2011, 09:04:35 pm
Part1

Still finding the occasional movie where Characteristics aren't being downloaded.  Lost Lake (http://www.allrovi.com/movies/movie/lost-lake-v323279), for example.

Hm, same problem as in the Synopsis, as well as the same solution:
~Characteristics~ and ~Themes~ or optionally ~mfCategory~  will be postponed until the end ParseMovie, follows attempts:
View AllRovi TEST Script

EDIT:
Sorry.

Since I was unable to load code (limit exceeded by more than 20.000 lines), for examples of how and what changes, attached AllRovi test script for testing.

EDIT2:
AllRovi test2 script attached, but the details are described here. (http://www.videodb.info/forum_en/index.php/topic,2496.msg13343.html#msg13343)


[attachment deleted by admin]
Title: Re: AllRovi movie script
Post by: rick.ca on August 16, 2011, 11:36:30 pm
Still finding the occasional movie where Characteristics aren't being downloaded.  Lost Lake (http://www.allrovi.com/movies/movie/lost-lake-v323279), for example.

If you don't mind, please test Ivek's modified script—for both the kind of movies you've had problems with (generally, ones for which AllRovi doesn't have complete information) and other more "normal" ones. I haven't got time to test, and want to be sure we're not breaking anything else with this. Thanks.
Title: Re: AllRovi movie script
Post by: RazorHall on August 17, 2011, 05:54:31 am
The test script fixed the problem for some movies, but not all.  Example: Edgar Allen Poe's Darkness (http://www.allrovi.com/movies/movie/edgar-allen-poes-darkness-v358651).
Title: Re: AllRovi movie script
Post by: rick.ca on August 17, 2011, 06:55:49 am
Thanks. I think we need to figure out what's actually going wrong with the logic of the script before releasing further changes. I assume the current version (0.1.1.2) is no worse than the previous version or Ivek's test version—that it only fails for some movies for which AllRovi does not have complete information).
Title: Re: AllRovi movie script
Post by: Ivek23 on August 17, 2011, 08:00:17 am
The test script fixed the problem for some movies, but not all.  Example: Edgar Allen Poe's Darkness (http://www.allrovi.com/movies/movie/edgar-allen-poes-darkness-v358651).

How in this case the current version works.

Thanks. I think we need to figure out what's actually going wrong with the logic of the script before releasing further changes. I assume the current version (0.1.1.2) is no worse than the previous version or Ivek's test version—that it only fails for some movies for which AllRovi does not have complete information).

Will cost us a few brain cells to figure out what is actually wrong with the logic of the script.

BTW:
I suggest that until we find solutions to these problems, current version is used, if it does not transfer all info data which are available for a particular movie, then a test version.

I hope that this combination is currently a solution for the time being to problems.
Title: Re: AllRovi movie script
Post by: Ivek23 on August 17, 2011, 02:30:14 pm
For many problems medicine is so, so it was also a problem with AllRovi script is also found medicine to solve problems.

So when I was looking for solutions and work tests for this one Edgar Allen Poe's Darkness  (http://www.allrovi.com/movies/movie/edgar-allen-poes-darkness-v358651) movie, which has caused me many problems and I almost gave up on this , I came up thought, what if I try the current AllRovi script only copy ~Characteristics~ and added to the end ParseMovie.
I was first surprised that the script is working at all, even greater surprise was when I have the above-mentioned film, as well as Lost Lake (http://www.allrovi.com/movies/movie/lost-lake-v323279) movie transferred all info available data.
Tested about 30 movie and fine work.

See images.
AllRovi test2 script attached.

[attachment deleted by admin]
Title: Re: AllRovi movie script
Post by: RazorHall on August 17, 2011, 03:11:45 pm
A little more progress there, but the Characteristics problem still shows up on a select few movies, such as Lustful Addiction (http://www.allrovi.com/movies/movie/lustful-addiction-v276283).
Title: Re: AllRovi movie script
Post by: rick.ca on August 17, 2011, 11:26:31 pm
I don't think moving routines around is the solution. I'm still not sure my solution works in all situations, but it seems to me the problem occurs when 'curPos' gets past the point in the page where the data exists (in this case, Characteristics). This can happen when the order of the data elements on the page change. Generally, the order does not change, but it can when all the usual data elements do not exist. So my solution is simply to reset 'curPos' to 'topPos' (introduced in an earlier version to solve the same issue for Rating and Genre). It seems to work for all the "problem" titles mentioned lately, and I haven't seen it break anything else. But...

Code: [Select]
//Save start position for Movie data
topPos := PosFrom('<div class="page-heading">', HTML, curPos)

...may not hold for those rare cases where what's normally a child page/tab doubles as the main page. Maybe I already have, but I'll test that once I recall an example of where we found that issue before. [Edit:] It seems I've already tested this, using Top Model (http://www.allrovi.com/movies/movie/top-model-v130496) and The Dallas Connection (http://www.allrovi.com/movies/movie/the-dallas-connection-v154289)—cases where 'Cast and Crew' serves as the main page. So I think the only concern would be variations on that theme we have not yet encountered. :-\

Also, there are two issues involved with titles like Lustful Addiction. First, the AllRovi search does not find the movie. The page can only be accessed by entering the URL. (See Adult titles are there, but not discoverable? (http://getsatisfaction.com/allrovi/topics/adult_titles_are_there_but_not_discoverable)) If the PVD record has the correct URL, the script should work fine. That is, Characteristics should now be retrieved.

All this from one of my few remaining brain cells. Please torture-test the attached. ;)

[attachment deleted by admin]
Title: Re: AllRovi movie script
Post by: RazorHall on August 18, 2011, 02:28:13 am
AllRovi 1.3 Candidate works perfectly on every title I've tried so far.  I'll let you know if any other weird cases should appear.  And thanks!  You guys are doing a terrific job.   :D
Title: Re: AllRovi movie script
Post by: rick.ca on August 18, 2011, 06:32:04 am
Thanks. But let's not get too excited. Most of my test titles came from you. Who knows what we'll hear from, say, the art film crowd... ;)
Title: Re: AllRovi movie script
Post by: Ivek23 on August 18, 2011, 02:58:41 pm
This movies
Stargate Universe [TV Series] (2009) (http://www.allrovi.com/movies/movie/stargate-universe-tv-series-v478052/cast_crew)
Stargate Atlantis [TV Series] (2004) (http://www.allrovi.com/movies/movie/stargate-atlantis-tv-series-v312306/cast_crew)
Stargate Infinity [Animated TV Series] (2002) (http://www.allrovi.com/movies/movie/stargate-infinity-animated-tv-series-v272518/cast_crew)
have something in common in the Cast & Crew tab, there is only the Crew section, but is not there Cast section, so AllRovi script did not pass the info data with Section Crew.

So I ParseCast adding pieces of code
Code: [Select]
//~Cast~ or ~ctActors~
//save entire Cast tab to custom memo field, as well as individual actors.

procedure ParseCast(HTML : String);
var
curPos, endPos : Integer;
TmpStr : String;
Name, Role, URL : String;
begin
curPos := Pos('<div class="tabset-content main-tab-pane">', HTML);
if curPos < 1 then

curPos := Pos('<div class="description-box">', HTML);
if curPos < 1 then
Exit;
 
TmpStr := '';
 ...

 ...
AddCustomFieldValueByName('Cast', TmpStr);


//~Crew~
curPos := Pos('<div class="profession-box">', HTML);
if curPos < 1 then
Exit;

TmpStr := '';
 ...

 ...
end;

AddCustomFieldValueByName('Crew', TmpStr);

end;


//~Releases~


and now pass the info data with Section Crew,
(edit:) or if the Cast section and Crew section both there.

EDIT
AllRovi 1.3 candidate 1 attached.

[attachment deleted by admin]
Title: Re: AllRovi movie script
Post by: rick.ca on August 18, 2011, 11:25:27 pm
Quote
AllRovi 1.3 candidate 1 attached.

I haven't tested it yet, but I'm curious about this...

Code: [Select]
//~Crew~curPos := Pos('<div class="profession-box">', HTML);
if curPos < 1 then
Exit;

Did you intend to disable the setting of curPos, or did you accidentally delete the line feed?

[Edit:] You must have—it doesn't work with this line active. I've removed it.
Title: Re: AllRovi movie script
Post by: rick.ca on August 19, 2011, 01:04:06 am
Quote
...have something in common in the Cast & Crew tab, there is only the Crew section, but is not there Cast section, so AllRovi script did not pass the info data with Section Crew.

Nice catch. This situation seems to be very common for TV Series. I hadn't noticed because I don't use AllRovi for Series. It's rare for movies, but not so uncommon for documentaries. The script seems to work for all of them.

And it works for all my test records, so this is it. I'm uploading version 1.3 now.
Title: Re: AllRovi movie script
Post by: Ivek23 on August 19, 2011, 06:05:57 am
Code: [Select]
//~Crew~curPos := Pos('<div class="profession-box">', HTML);
if curPos < 1 then
Exit;
It was my mistake.

Code: [Select]
//~Crew~

curPos := Pos('<div class="profession-box">', HTML);
if curPos < 1 then
Exit;
Meant it was such a piece of code.

Code: [Select]
//~Crew~

if curPos < 1 then
Exit;
Even this piece of code works great, I have a feeling that the script works even better with this part of code than with the part above the proper codes meant, and it was quite right that it was this one piece of code
Code: [Select]
curPos := Pos('<div class="profession-box">', HTML);removed. I tested with this part of the code
Code: [Select]
//~Crew~

if curPos < 1 then
Exit;
test movies, series and documentaries, is really perfect works. so sorry if I just now spoke up and answered the question.

Title: Re: AllRovi movie script
Post by: rick.ca on August 19, 2011, 10:26:30 am
Well, we're not programmers. If some things didn't work by accident, where would we be? ;)
Title: Re: AllRovi movie script
Post by: Ivek23 on August 19, 2011, 08:23:20 pm
Well, we're not programmers. If some things didn't work by accident, where would we be? ;)
I agree.
Title: Re: AllRovi movie script
Post by: RazorHall on August 20, 2011, 04:35:51 pm
Who knows what we'll hear from, say, the art film crowd... ;)

Oh, I like those too and have plenty of them in my collection.  Just haven't run into any problems with them yet.   8)
Title: Re: AllRovi movie script
Post by: Ivek23 on December 13, 2012, 10:42:46 am
I made a slight adjustment in ParseSearchResults section.
I have this piece of code:
Code: [Select]
if Pos('Search Results for', HTML) > 0 then begin
ParseSearchResults(HTML);
Result := prList;
Exit;
end else

if (Mode = 0) AND (Pos('<h2>cast</h2>', HTML) > 0) then begin
ParseMovie(URL, HTML);
LogMessage('Getting main details from Cast & Crew page...');
end else
if (Mode = 0) AND (Pos('<strong>review</strong>', HTML) > 0) then begin
ParseMovie(URL, HTML);
LogMessage('Getting main details from Review page...');
end else
if (Mode = 0) AND (Pos('<h2>releases</h2>', HTML) > 0) then
ParseMovie(URL, HTML)
else
if Pos('- Cast, Reviews, Summary, and Awards - AllRovi</title>', HTML) > 0 then
ParseMovie(URL, HTML)
else
if Pos('- Review - AllRovi</title>', HTML) > 0 then
ParseReview(HTML)
else
if Pos('- Cast and Crew - AllRovi</title>', HTML) > 0 then
ParseCast(HTML)
else
if Pos('- Releases - AllRovi</title>', HTML) > 0 then
ParseDVDReleases(HTML);

Mode := NextMode(Mode);

replace this part of code:
Code: [Select]
if Pos('Search Results for', HTML) > 0 then begin
ParseSearchResults(HTML);
Result := prList;
Exit;
end else

if (Mode = 0) AND (Pos('<span>cast & crew</span></a>', HTML) > 0) then begin
ParseMovie(URL, HTML);
LogMessage('Getting main details from Cast & Crew page...');
end else
if (Mode = 0) AND (Pos('<span>review</span></a>', HTML) > 0) then begin
ParseMovie(URL, HTML);
LogMessage('Getting main details from Review page...');
end else
if (Mode = 0) AND (Pos('<span>releases</span></a>', HTML) > 0) then begin
ParseMovie(URL, HTML);
LogMessage('Getting main details from Releases page...');
end else
if (Mode = 0) AND (Pos('<h2>cast</h2>', HTML) > 0) then begin
ParseMovie(URL, HTML);
LogMessage('Getting main details from Cast & Crew page...');
end else
if (Mode = 0) AND (Pos('<strong>review</strong>', HTML) > 0) then begin
ParseMovie(URL, HTML);
LogMessage('Getting main details from Review page...');
end else
if (Mode = 0) AND (Pos('<h2>releases</h2>', HTML) > 0) then
ParseMovie(URL, HTML)
else
if Pos('- Cast, Reviews, Summary, and Awards - AllRovi</title>', HTML) > 0 then
ParseMovie(URL, HTML)
else
if Pos('- Review - AllRovi</title>', HTML) > 0 then
ParseReview(HTML)
else
if Pos('- Cast and Crew - AllRovi</title>', HTML) > 0 then
ParseCast(HTML)
else
if Pos('- Releases - AllRovi</title>', HTML) > 0 then
ParseDVDReleases(HTML);

Mode := NextMode(Mode);

Now in the movie, with only definite information is now enabled data transfer. Previous version of the script, these data did not downloaded.

Notice:

For once this correction is not valid, the correction followed later.
Remains the first method, because this correction in the script (if existing url addresses already stored in the database) , then script does
not correctly work.


Correction in this post:
http://www.videodb.info/forum_en/index.php/topic,2496.msg16250.html#msg16250
Title: Re: AllRovi movie script
Post by: Ivek23 on December 13, 2012, 07:27:26 pm
~Moods~ from 'Characteristics' and save ~Mood links~ if available

I added some url links (below is the new list of url links) to AllMovie mood pages.

Code: [Select]
//Pull ~Moods~ from 'Characteristics' and save ~Mood links~ if available
TmpStr2 := '';
TmpStr3 := '';
if Pos('A Good Cry', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'A Good Cry';
if Pos('A Good Cry', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://allmovie.com/explore/mood/a-good-cry-2028">A Good Cry</link>';
TmpStr := StringReplace(TmpStr, 'A Good Cry', '', true, true, true);
if Pos('A World of Its Own', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'A World of Its Own';
if Pos('A World of Its Own', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + 'A World of Its Own';
TmpStr := StringReplace(TmpStr, 'A World of Its Own', '', true, true, true);
if Pos('Abandon All Hope', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Abandon All Hope';
if Pos('Abandon All Hope', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/abandon-all-hope-2066">Abandon All Hope</link>';
TmpStr := StringReplace(TmpStr, 'Abandon All Hope', '', true, true, true);
if Pos('Adrenaline Rush', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Adrenaline Rush';
if Pos('Adrenaline Rush', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/adrenaline-rush-2026">Adrenaline Rush</link>';
TmpStr := StringReplace(TmpStr, 'Adrenaline Rush', '', true, true, true);
if Pos('Angsty', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Angsty';
if Pos('Angsty', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + 'Angsty';
TmpStr := StringReplace(TmpStr, 'Angsty', '', true, true, true);
if Pos('Bad Taste', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Bad Taste';
if Pos('Bad Taste', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/bad-taste-2031">Bad Taste</link>';
TmpStr := StringReplace(TmpStr, 'Bad Taste', '', true, true, true);
if Pos('Blood and Gore', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Blood and Gore';
if Pos('Blood and Gore', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/blood-and-gore-260">Blood and Gore</link>';
TmpStr := StringReplace(TmpStr, 'Blood and Gore', '', true, true, true);
if Pos('Button Pushers', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Button Pushers';
if Pos('Button Pushers', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + 'Button Pushers';
TmpStr := StringReplace(TmpStr, 'Button Pushers', '', true, true, true);
if Pos('Carnal Knowledge', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Carnal Knowledge';
if Pos('Carnal Knowledge', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/carnal-knowledge-262">Carnal Knowledge</link>';
TmpStr := StringReplace(TmpStr, 'Carnal Knowledge', '', true, true, true);
if Pos('Comedy on the Edge', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Comedy on the Edge';
if Pos('Comedy on the Edge', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/comedy-on-the-edge-2030">Comedy on the Edge</link>';
TmpStr := StringReplace(TmpStr, 'Comedy on the Edge', '', true, true, true);
if Pos('Estrogen Shot', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Estrogen Shot';
if Pos('Estrogen Shot', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + 'Estrogen Shot';
TmpStr := StringReplace(TmpStr, 'Estrogen Shot', '', true, true, true);
if Pos('Eyepoppers', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Eyepoppers';
if Pos('Eyepoppers', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + 'Eyepoppers';
TmpStr := StringReplace(TmpStr, 'Eyepoppers', '', true, true, true);
if Pos('Fantastic Reality', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Fantastic Reality';
if Pos('Fantastic Reality', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/fantastic-reality-2024">Fantastic Reality</link>';
TmpStr := StringReplace(TmpStr, 'Fantastic Reality', '', true, true, true);
if Pos('Flames of Passion', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Flames of Passion';
if Pos('Flames of Passion', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/flames-of-passion-262">Flames of Passion</link>';
TmpStr := StringReplace(TmpStr, 'Flames of Passion', '', true, true, true);
if Pos('Food for Thought', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Food for Thought';
if Pos('Food for Thought', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/food-for-thought-2032">Food for Thought</link>';
TmpStr := StringReplace(TmpStr, 'Food for Thought', '', true, true, true);
if Pos('For Love and Country', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'For Love of Country';
if Pos('For Love and Country', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/god-and-country-257">For Love and Country</link>';
TmpStr := StringReplace(TmpStr, 'For Love and Country', '', true, true, true);
if Pos('God and Country', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'God and Country';
if Pos('God and Country', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/god-and-country-257">God and Country</link>';
TmpStr := StringReplace(TmpStr, 'God and Country', '', true, true, true);
if Pos('Gutbusters', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Gutbusters';
if Pos('Gutbusters', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + 'Gutbusters';
TmpStr := StringReplace(TmpStr, 'Gutbusters', '', true, true, true);
if Pos('Head Trips', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Head Trips';
if Pos('Head Trips', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://allmovie.com/explore/mood/head-trips-2035">Head Trips</link>';
TmpStr := StringReplace(TmpStr, 'Head Trips', '', true, true, true);
if Pos('High on Emotion', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'High on Emotion';
if Pos('High on Emotion', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/high-on-emotion-2028">High on Emotion</link>';
TmpStr := StringReplace(TmpStr, 'High on Emotion', '', true, true, true);
if Pos('In a Minor Key', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'In a Minor Key';
if Pos('In a Minor Key', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/in-a-minor-key-269">In a Minor Key</link>';
TmpStr := StringReplace(TmpStr, 'In a Minor Key', '', true, true, true);
if Pos('In the Mood for Love', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'In the Mood for Love';
if Pos('In the Mood for Love', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/in-the-mood-for-love-2027">In the Mood for Love</link>';
TmpStr := StringReplace(TmpStr, 'In the Mood for Love', '', true, true, true);
if Pos('Just for Fun', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Just for Fun';
if Pos('Just for Fun', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/just-for-fun-2029">Just for Fun</link>';
TmpStr := StringReplace(TmpStr, 'Just for Fun', '', true, true, true);
if Pos('Just for Laughs', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Just for Laughs';
if Pos('Just for Laughs', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/just-for-laughs-2029">Just for Laughs</link>';
TmpStr := StringReplace(TmpStr, 'Just for Laughs', '', true, true, true);
if Pos('Memory Lane', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Memory Lane';
if Pos('Memory Lane', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/memory-lane-2025">Memory Lane</link>';
TmpStr := StringReplace(TmpStr, 'Memory Lane', '', true, true, true);
if Pos('Mindbenders', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Mindbenders';
if Pos('Mindbenders', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/mindbenders-2035">Mindbenders</link>';
TmpStr := StringReplace(TmpStr, 'Mindbenders', '', true, true, true);
if Pos('Mood Enhancers', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Mood Enhancers';
if Pos('Mood Enhancers', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/mood-enhancers-2036">Mood Enhancers</link>';
TmpStr := StringReplace(TmpStr, 'Mood Enhancers', '', true, true, true);
if Pos('Nail-biters', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Nail-biters';
if Pos('Nail-biters', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/nail-biters-2037">'Nail-biters</link>';
TmpStr := StringReplace(TmpStr, 'Nail-biters', '', true, true, true);
if Pos('Off the Beaten Path', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Off the Beaten Path';
if Pos('Off the Beaten Path', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + 'Off the Beaten Path';
TmpStr := StringReplace(TmpStr, 'Off the Beaten Path', '', true, true, true);
if Pos('Only Human', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Only Human';
if Pos('Only Human', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/only-human-264">Only Human</link>';
TmpStr := StringReplace(TmpStr, 'Only Human', '', true, true, true);
if Pos('Other Dimensions', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Other Dimensions';
if Pos('Other Dimensions', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/other-dimensions-2038">Other Dimensions</link>';
TmpStr := StringReplace(TmpStr, 'Other Dimensions', '', true, true, true);
if Pos('Pick-Me-Ups', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Pick-Me-Ups';
if Pos('Pick-Me-Ups', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/pick-me-ups-2036">Pick-Me-Ups</link>';
TmpStr := StringReplace(TmpStr, 'Pick-Me-Ups', '', true, true, true);
if Pos('Slow Burn', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Slow Burn';
if Pos('Slow Burn', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + 'Slow Burn';
TmpStr := StringReplace(TmpStr, 'Slow Burn', '', true, true, true);
if Pos('Spellbinders', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Spellbinders';
if Pos('Spellbinders', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/spellbinders-265">Spellbinders</link>';
TmpStr := StringReplace(TmpStr, 'Spellbinders', '', true, true, true);
if Pos('Strictly Speaking', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Strictly Speaking';
if Pos('Strictly Speaking', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/strictly-speaking-2034">Strictly Speaking</link>';
TmpStr := StringReplace(TmpStr, 'Strictly Speaking', '', true, true, true);
if Pos('Thrill Rides', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Thrill Rides';
if Pos('Thrill Rides', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/thrill-rides-2026">Thrill Rides</link>';
TmpStr := StringReplace(TmpStr, 'Thrill Rides', '', true, true, true);
if Pos('Tough Guys', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Tough Guys';
if Pos('Tough Guys', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/tough-guys-259">Tough Guys</link>';
TmpStr := StringReplace(TmpStr, 'Tough Guys', '', true, true, true);
if Pos('Trashy', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Trashy';
if Pos('Trashy', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/trashy-2031">Trashy</link>';
TmpStr := StringReplace(TmpStr, 'Trashy', '', true, true, true);
if Pos('Triumph of the Geeks', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Triumph of the Geeks';
if Pos('Triumph of the Geeks', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + 'Triumph of the Geeks';
TmpStr := StringReplace(TmpStr, 'Triumph of the Geeks', '', true, true, true);
if Pos('Triumph of the Spirit', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Triumph of the Spirit';
if Pos('Triumph of the Spirit', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + '<link url="http://web.archive.org/web/http://www.allmovie.com/explore/mood/triumph-of-the-spirit-2033">Triumph of the Spirit</link>';
TmpStr := StringReplace(TmpStr, 'Triumph of the Spirit', '', true, true, true);
if Pos('Uncomfortable Viewing', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Uncomfortable Viewing';
if Pos('Uncomfortable Viewing', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + 'Uncomfortable Viewing';
TmpStr := StringReplace(TmpStr, 'Uncomfortable Viewing', '', true, true, true);
if Pos('Unloveables', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Unloveables';
if Pos('Unloveables', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + 'Unloveables';
TmpStr := StringReplace(TmpStr, 'Unloveables', '', true, true, true);
if Pos('Young and Old Alike', TmpStr) > 0 then TmpStr2 := TmpStr2 + ', ' + 'Young and Old Alike';
if Pos('Young and Old Alike', TmpStr) > 0 then TmpStr3 := TmpStr3 + ', ' + 'Young and Old Alike';
TmpStr := StringReplace(TmpStr, 'Young and Old Alike', '', true, true, true);

TmpStr := StringReplace(TmpStr, ' A ', ' a ', true, false, true);
TmpStr := StringReplace(TmpStr, ' An ', ' an ', true, false, true);
TmpStr := StringReplace(TmpStr, ' And ', ' and ', true, false, true);
TmpStr := StringReplace(TmpStr, ' & ', ' and ', true, false, true);
TmpStr := StringReplace(TmpStr, ' By ', ' by ', true, false, true);
TmpStr := StringReplace(TmpStr, ' For ', ' for ', true, false, true);
TmpStr := StringReplace(TmpStr, ' From ', ' from ', true, false, true);
TmpStr := StringReplace(TmpStr, ' In ', ' in ', true, false, true);
TmpStr := StringReplace(TmpStr, ' Into ', ' into ', true, false, true);
TmpStr := StringReplace(TmpStr, ' Of ', ' of ', true, false, true);
TmpStr := StringReplace(TmpStr, ' On ', ' on ', true, false, true);
TmpStr := StringReplace(TmpStr, ' Or ', ' or ', true, false, true);
TmpStr := StringReplace(TmpStr, ' The ', ' the ', true, false, true);
TmpStr := StringReplace(TmpStr, ' To ', ' to ', true, false, true);
TmpStr := StringReplace(TmpStr, ' With ', ' with ', true, false, true);
TmpStr := StringReplace(TmpStr, ' ,', '', true, true, true);
if Pos(', ', TmpStr) = 1 then Delete(TmpStr, 1, 2);
AddCustomFieldValueByName('Characteristics', TmpStr);

AddCustomFieldValueByName('Attributes', TmpStr1);

AddCustomFieldValueByName('Moods', TmpStr2);

Delete(TmpStr3, 1, 2);
AddCustomFieldValueByName('Mood links', TmpStr3);
Title: Re: AllRovi movie script
Post by: Ivek23 on December 28, 2012, 09:28:22 am
Correction relates to this post:
http://www.videodb.info/forum_en/index.php/topic,2496.msg16185.html#msg16185


I made a slight adjustment in ParsePage function (ParseSearchResults section).

I have this piece of code:
Code: [Select]
if Pos('Search Results for', HTML) > 0 then begin
ParseSearchResults(HTML);
Result := prList;
Exit;
end else

if (Mode = 0) AND (Pos('<h2>cast</h2>', HTML) > 0) then begin
ParseMovie(URL, HTML);
LogMessage('Getting main details from Cast & Crew page...');
end else
if (Mode = 0) AND (Pos('<strong>review</strong>', HTML) > 0) then begin
ParseMovie(URL, HTML);
LogMessage('Getting main details from Review page...');
end else
if (Mode = 0) AND (Pos('<h2>releases</h2>', HTML) > 0) then
ParseMovie(URL, HTML)
else
if Pos('- Cast, Reviews, Summary, and Awards - AllRovi</title>', HTML) > 0 then
ParseMovie(URL, HTML)
else
if Pos('- Review - AllRovi</title>', HTML) > 0 then
ParseReview(HTML)
else
if Pos('- Cast and Crew - AllRovi</title>', HTML) > 0 then
ParseCast(HTML)
else
if Pos('- Releases - AllRovi</title>', HTML) > 0 then
ParseDVDReleases(HTML);

Mode := NextMode(Mode);

No, not with this part of code:
replace this part of code:
Code: [Select]
if Pos('Search Results for', HTML) > 0 then begin
ParseSearchResults(HTML);
Result := prList;
Exit;
end else

if (Mode = 0) AND (Pos('<span>cast & crew</span></a>', HTML) > 0) then begin
ParseMovie(URL, HTML);
LogMessage('Getting main details from Cast & Crew page...');
end else
if (Mode = 0) AND (Pos('<span>review</span></a>', HTML) > 0) then begin
ParseMovie(URL, HTML);
LogMessage('Getting main details from Review page...');
end else
if (Mode = 0) AND (Pos('<span>releases</span></a>', HTML) > 0) then begin
ParseMovie(URL, HTML);
LogMessage('Getting main details from Releases page...');
end else
if (Mode = 0) AND (Pos('<h2>cast</h2>', HTML) > 0) then begin
ParseMovie(URL, HTML);
LogMessage('Getting main details from Cast & Crew page...');
end else
if (Mode = 0) AND (Pos('<strong>review</strong>', HTML) > 0) then begin
ParseMovie(URL, HTML);
LogMessage('Getting main details from Review page...');
end else
if (Mode = 0) AND (Pos('<h2>releases</h2>', HTML) > 0) then
ParseMovie(URL, HTML)
else
if Pos('- Cast, Reviews, Summary, and Awards - AllRovi</title>', HTML) > 0 then
ParseMovie(URL, HTML)
else
if Pos('- Review - AllRovi</title>', HTML) > 0 then
ParseReview(HTML)
else
if Pos('- Cast and Crew - AllRovi</title>', HTML) > 0 then
ParseCast(HTML)
else
if Pos('- Releases - AllRovi</title>', HTML) > 0 then
ParseDVDReleases(HTML);

Mode := NextMode(Mode);
but

replace this part of code:

Code: [Select]
if Pos('Search Results for', HTML) > 0 then begin
ParseSearchResults(HTML);
Result := prList;
Exit;
end else

if (Mode = 0) AND (Pos('<h2>cast</h2>', HTML) > 0) then begin
ParseMovie(URL, HTML);
LogMessage('Getting main details from Cast & Crew page...');
end else
if (Mode = 0) AND (Pos('<h2>crew</h2>', HTML) > 0) then begin
ParseMovie(URL, HTML);
LogMessage('Getting main details from Cast & Crew page...');
end else
if (Mode = 0) AND (Pos('<strong>review</strong>', HTML) > 0) then begin
ParseMovie(URL, HTML);
LogMessage('Getting main details from Review page...');
end else
if (Mode = 0) AND (Pos('<h2>releases</h2>', HTML) > 0) then
ParseMovie(URL, HTML)
else
if Pos('- Cast, Reviews, Summary, and Awards - AllRovi</title>', HTML) > 0 then
ParseMovie(URL, HTML)
else
if Pos('- Review - AllRovi</title>', HTML) > 0 then
ParseReview(HTML)
else
if Pos('- Cast and Crew - AllRovi</title>', HTML) > 0 then
ParseCast(HTML)
else
if Pos('- Releases - AllRovi</title>', HTML) > 0 then
ParseDVDReleases(HTML);

Mode := NextMode(Mode);

Now in the movie, with only definite information is now enabled data transfer. Previous version of the script, these data did not downloaded.
Title: Re: AllRovi movie script
Post by: Ivek23 on January 07, 2014, 11:56:04 am
Notice:

AllRovi script does not work anymore.