Personal Video Database
English => Development => Scripts and Templates => Topic started by: raldo on March 25, 2009, 05:30:36 pm
-
I don't know if anyone here uses J River Media Center (http://www.jrmediacenter.com/), but here is an export template that will output to an MPL file. MPL is the main database format that Media Center (MC) uses. MC is great as an organisational/playback/home theatre setup, but lacks online metadata lookup for movies, so this is where PVD comes in for me Smiley
First of all, set up the export template in PVD-
go to tools > preferences > plugins > plain text/export > add > (call it what you want) > and then copy and pase the following:
[...]
When exporting using this method, each movie record results in one exported record. When a movie/record contains multiple files, the filename is comprised of pipe "|" separated filenames.
Is there a way using export templates to ouptput multiple export records where each record represents only one file?
The reason why I'm asking, is that on the J. River MC side, the MPL file is imported incorrectly for movies with multiple files.
-
Is there a way using export templates to output multiple export records where each record represents only one file?
It's been a while since I've done this, but I have a vague recollection I had to edit the export file to deal with this. I suppose the easiest way to do that is to eliminate all but the first file reference—perhaps it's not necessary to tag all the files of a multiple file movie in MC.
BTW, what version of PVD are you using? Are you using darichman's script, or have you updated for use with 0.9.9.4?
-
PVD: 0.9.8.20
script: 0.5.0.6
Why?
Using just one filepath would naturally solve the problem to a certain extent, as you explain. It would be a lot of manual work, though.
-
Why?
No reason, now that I remind myself the export function doesn't yet work in the 0.9.9.4 (it needs to be updated in respect of how fields are now referenced by name). I knew there was some reason why I hadn't done this in a while. :-\ It seems to be a snail's race between that getting done, an MC improving Theater View so the data is actually worth having there.
It would be a lot of manual work, though.
It would require a text editor that uses regex, and the regex for finding and deleting the text between the first "|" and "</" of the filename tag.
-
[...]
It would require a text editor that uses regex, and the regex for finding and deleting the text between the first "|" and "</" of the filename tag.
Yes, that's possible. Still, It's cumbersome.
It's a pity because I have a setup that's relatively simple to use now:
Open PVD, search folders and export. The export launches a batch file which again launches MC with import playlist command line options. Done.
-
Rick.ca:
There has been a lot of discussion on the JRMC forums regarding feasible methods on obtaining meta data for video.
What, would you say, is the possibility of somehow more closely integrating PVD with MC?
-
It's a pity because I have a setup that's relatively simple to use now.
That's slick. Would you like to share your routine? I'm also curious about how you handle the list fields.
What, would you say, is the possibility of somehow more closely integrating PVD with MC?
Except for the problem you mention, it sounds like you've come pretty close to an ideal. Even if it has to be done manually, it only takes a few minutes. Beyond that, I can only offer some conflicting thoughts...
Since PVD uses a Firebird database, there's a possibility of the data being pulled directly from the database. This would probably be child's play for MC developers, but they're not going to do it for a handful of users using PVD.
Most people willing to make the effort to collect the information are going to use custom fields, use standard fields for other uses, and have differing preferences as to how data elements are handled in PVD and MC. Accommodating all the possible variations makes closer integration more difficult. In fact, with this in mind, the flexibility of exporting to XML starts to look pretty good.
A user with sufficient programming knowledge could write an application that reads the database and writes a MPL file. The advantage of that is it could be scheduled to run at regular intervals, and MC could be configured to import it regularly. The fact that both the export and import would run automatically would make it seem the two applications were integrated.
While it doesn't seem like a big deal, there is user data created in MC (e.g., date played, rating) that has no way of being recorded (automatically) in PVD. As a practical matter, I suppose the best way to deal with such things is to import the PVD counterparts into separate fields—for comparison to the MC values. In MC you would have the choice of displaying one, the other, or a calculated result, and would be able to report at any time what needs to be updated in PVD to bring it in sync.
Regardless of the integration issues, I'm convinced MC will never have an information gathering ability of it's own that will come close to what PVD can do. So, hopefully, we can work through the integration issues. Meanwhile, I'm enjoy using PVD with my "Theater View" skin to "collect, organize and play" my movies. ;)
-
[...]
That's slick. Would you like to share your routine? I'm also curious about how you handle the list fields.
[...]
I'll post the scripts when I get back home later on today.
I assume you're talking about semi colon (after replace in output script) delimited fields? I have posted some suggestions over at the mc forums regarding this. http://yabb.jriver.com/interact/index.php?topic=51208.0 (http://yabb.jriver.com/interact/index.php?topic=51208.0). It'll be interesting to see if for example "Director" (MC) will be changed to accomodate this.
[...]
Except for the problem you mention, it sounds like you've come pretty close to an ideal. Even if it has to be done manually, it only takes a few minutes. Beyond that, I can only offer some conflicting thoughts...
Since PVD uses a Firebird database, there's a possibility of the data being pulled directly from the database. This would probably be child's play for MC developers, but they're not going to do it for a handful of users using PVD.
Most people willing to make the effort to collect the information are going to use custom fields, use standard fields for other uses, and have differing preferences as to how data elements are handled in PVD and MC. Accommodating all the possible variations makes closer integration more difficult. In fact, with this in mind, the flexibility of exporting to XML starts to look pretty good.
A user with sufficient programming knowledge could write an application that reads the database and writes a MPL file. The advantage of that is it could be scheduled to run at regular intervals, and MC could be configured to import it regularly. The fact that both the export and import would run automatically would make it seem the two applications were integrated.
While it doesn't seem like a big deal, there is user data created in MC (e.g., date played, rating) that has no way of being recorded (automatically) in PVD. As a practical matter, I suppose the best way to deal with such things is to import the PVD counterparts into separate fields—for comparison to the MC values. In MC you would have the choice of displaying one, the other, or a calculated result, and would be able to report at any time what needs to be updated in PVD to bring it in sync.
Regardless of the integration issues, I'm convinced MC will never have an information gathering ability of it's own that will come close to what PVD can do. So, hopefully, we can work through the integration issues. Meanwhile, I'm enjoy using PVD with my "Theater View" skin to "collect, organize and play" my movies. ;)
But what about the following solution which requires "smaller" changes on the PVD side? And practically NO changes on the JRMC side:
o Use send to external in MC (with filename as parameter) to pass parameters to PVD (Similar to how JRMC can receive parameters)
o Implement a silent mode on the PVD side, where search, update and export are done "behind the curtains".
o PVD's export function "runs" the exported MPL file
o -- OR even better: PVD sends a core command to MC with mpl filename attached. Doesn't require OS to trigger import
o JRMC dialog pops up when import has finished.
-
[...]Would you like to share your routine? [...]
Export template:
%OPTIONS%
replace="&<->&"
replace="<<-><"
replace="><->>"
replace=",<->;"
filter="MPL Files|*.mpl"
encoding="UTF8"
%OPTIONS%
%HEAD%
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<MPL Version="2.0" Title="PVD->MC Export file">
%HEAD%
<Item>
{%value=27 template="pages\files.ptm"}
<Field Name="Year">{%value=5}</Field>
<Field Name="Genre">{%value=44}</Field>
<Field Name="MPAA Rating">{%value=6}</Field>
<Field Name="Actors">{%value=52}</Field>
<Field Name="Director">{%value=53}</Field>
<Field Name="Producer">{%value=56}</Field>
<Field Name="Screenwriter">{%value=54}</Field>
<Field Name="Original Title">{%value=3}</Field>
<Field Name="Country">{%value=45}</Field>
<Field Name="Tag Line">{%value=14}</Field>
<Field Name="Studios">{%value=50}</Field>
<Field Name="Description">{%value=15}</Field>
<Field Name="Season">{%value=42}</Field>
<Field Name="Episode">{%value=41}</Field>
<Field Name="Keywords">{%value=51}</Field>
<Field Name="IMDBRating">{%value=9}</Field>
<Field Name="Composer">{%value=55}</Field>
</Item>
%FOOT%
</MPL>
%FOOT%
And then the template for filenames, should be placed under [...]\PersonalVideo..\Plugins\template\pages
<Field Name="Filename">{%value}</Field>
This yields duplicate filenames for each record in the mpl file, but MC manages to import the records anyways. The good news here is that movies with more than one file get metadata for one of the files!
In the configuration dialog for the plain text template, tick "Open file upon completion". The new export template is also configured as "Use As Default".
In PVD, all folders that contain movies are selected in the File Scanner Group of the Tools\preferences\folders
I then associate .mpl files with a batch (.cmd) file that has the following text (create a file called "ImportMc.cmd"):
mc13.exe /Import %1
File types can be associated by right clicking the file, selecting properties and then pressing "Change" in the General tab. Then press browse and point to the batch file.
When this is correctly set up I can import new files into mc by:
o PVD: Tools\Scan folders. One may have to press ok some times.
o PVD: Tools\Export
-
To split files use this code in main template file:
Files:
{%value=path template="pages\files.ptm"}
And this in files.ptm
{%value}
P.S. You will need to use field number instead of "path" if you use 0.9.8.20
-
To split files use this code in main template file:
Files:
{%value=path template="pages\files.ptm"}
And this in files.ptm
{%value}
P.S. You will need to use field number instead of "path" if you use 0.9.8.20
Nostra:
This is a step forward but consider this: In the resulting MPL file, I would like one <item> section per File. When I do what you suggest above, I get one item section where the filenames appear twice.
I tried moving the Item section into files.ptm, but then %value=xx didn't parse into the resulting mpl file.
Can a child template somehow inherit the values from the parent template?
-
I've found a solution which works better than before. See Edit above...
-
You want to get only one (first?) file for arecord or what?
If so then define the template link like this:
{%value=27 template="pages\files.ptm" maxvalues="1"}
-
I assume you're talking about semi colon (after replace in output script) delimited fields?
Yes. I see you just replace all commas with semi colons. I used a separate export to do that only for list fields—so it wouldn't replace commas in fields like Description.
It'll be interesting to see if for example "Director" (MC) will be changed to accommodate this.
I attempted to explain here (http://yabb.jriver.com/interact/index.php?topic=51105.msg348943#msg348943) some new field types that would be required to properly handle movie information, but there's no sign yet they "get it."
I've found a solution which works...
Well done! So, does this achieve a level of integration you're satisfied with, or do you think there might be a better solution?
I hope the 0.9.9 export function is completed/fixed soon—I'd like to give this a try. Or at least be ready when MC gets around to providing the field types we need.
-
[..]
Yes. I see you just replace all commas with semi colons. I used a separate export to do that only for list fields—so it wouldn't replace commas in fields like Description.
[...]
Yeah, I guess I can live with some extra semicolons.. Since the MPL file can be reimported, I guess I can do it all over again if those semicolons start bothering me!
[..]
I attempted to explain here (http://yabb.jriver.com/interact/index.php?topic=51105.msg348943#msg348943) some new field types that would be required to properly handle movie information, but there's no sign yet they "get it."
[...]
Notice how JRMC did the GUI design in cooperation with the users on the forum. I'm pretty sure they'll come around with a proper list of meta data for movies. As far as I understand, they're also working on theater view meta display view for movies.
It'd be interesting to know how the various online databases overlap and differ wrt. meta data. Has anyone done a survey?! The intersection between these services + some of the "Good differences" I guess should be the standard for what they end up with...
So, does this achieve a level of integration you're satisfied with, or do you think there might be a better solution?
[...]
It looks like this is as far as we can get without code changes on either side. It's pretty good, I'd say...
The best solution, of course, would be a plugin on the JRMC side with direct communication to PVD.
-
You want to get only one (first?) file for arecord or what?
If so then define the template link like this:
{%value=27 template="pages\files.ptm" maxvalues="1"}
maxvalues="1" results in the number two filename (alphabetically) being exported. I think the solution above is good enough for now...
Nostra, are you the owner of PVD? If yes, would you be interested in cooperating towards making a JRMC plugin where PVD serves JRMC's needs for meta data? JRMC has a pretty comprehensive API, look here: http://wiki.jrmediacenter.com/index.php/Media_Center_Automation (http://wiki.jrmediacenter.com/index.php/Media_Center_Automation).
On your side, I'd guess you'd have to develop another plugin type, a database Server of some sort. I work as a programmer, and I'd be interested in trying something like this.
-
Yeah, I guess I can live with some extra semicolons.. Since the MPL file can be reimported, I guess I can do it all over again if those semicolons start bothering me!
You could just use two templates—one for list fields, one for other. Now that you got the export/import down to one automated step, surely it would be twice as gratifying to run two of them! ;)
I'm pretty sure they'll come around with a proper list of meta data for movies. As far as I understand, they're also working on theater view meta display view for movies.
Yes, they've made it clear they will come up with something for Theater View. But when it comes to making changes to handle different types of data (e.g., ordered lists of [name] - [role]/[function] for credits) and they ask for a list of fields needed for movies, I have to wonder if they really understand the unique requirements for movies. MC already allows custom fields, so we don't really need them to add any "standard" fields for movies. What we need are a few new field types that do not currently exist.
It'd be interesting to know how the various online databases overlap and differ wrt. meta data. Has anyone done a survey?! The intersection between these services + some of the "Good differences" I guess should be the standard for what they end up with...
Actually, my primary concern is many MC users, and the developers, may think this is what needs to happen. My experience here tells me that approach will only satisfy those who don't really care much about the information, as long as it "appears" automatically. I wouldn't bother arguing that is "wrong," as a majority of MC users may be quite happy with that. But my interest is in collecting and maintaining my own personal set of information about movies. It comes from several sources, and I'm sure it's substantially different than what anyone else is collecting. And it's the only information I'm interested in seeing in MC. My hope is more MC users will come to appreciate using something like PVD—even if just as a tool for collecting the meta data—is a better solution than any one-size-fits-all approach. Then, maybe they'll find a way to accommodate both approaches.
-
I work as a programmer, and I'd be interested in trying something like this.
I'm having trouble keeping up with you... I should think anyone willing to augment our overworked Development Division is most welcome. ;)
Excuse me if I'm clueless about such things, but wouldn't your MC plugin just access the Firebird database directly? Why would you need a plugin on the PVD side? To enable the update of PVD from MC, I suppose. Would it be possible to defer this aspect if nostra is to busy to do anything immediately on this end? Could it still be a two-way update—as long as PVD is not running at the same time? And I don't know if there are any meaningful parallels, but have you looked at Kroozbox - PVD to media theatre solution (http://www.videodb.info/forum_en/index.php?topic=1031.0)?
I imagine this would be something that would necessarily determine all the fields used in a particular PVD database, and facilitate the mapping of any or all those fields to fields in a particular MC database. Right?
-
types of data (e.g., ordered lists of [name] - [role]/[function] for credits) and they ask for a list of fields needed for movies, I have to
Do you mean the annoying reordering of lists into alphabetical order? I agree that there should be another ordered list type.
I don't understand the syntax "[name] - [role]/[function]". Is this a new field type?
Then, maybe they'll find a way to accommodate both approaches.
Yes, I agree with you on this.
Do you want to start a new thread on this in the MC forums? The timing is correct now...
-
Excuse me if I'm clueless about such things, but wouldn't your MC plugin just access the Firebird database directly? Why would you need a plugin on the PVD side? To enable the update of PVD from MC, I suppose.
Yup, that may be a way to access the data. The reason why I think something must be done on the PVD side is, as you suggest, that MC needs to be able to tell PVD that action must be taken, I.e. update the database.
I was just suggesting a generic approach ("New Plugin Type") but a simpler solution may work as well.
-
Do you mean the annoying reordering of lists into alphabetical order?
Yes. It's critical that lists of credits remain in credits order.
I don't understand the syntax "[name] - [role]/[function]". Is this a new field type?
Yes. That's just my way of describing a generic format for all credits. More specifically, it would be "[name] - [role]" for actors, and "[name] - [function, position or job title]" for production credits. It could then be used for any type of credit. For example, you could chose not to have a separate Director field, saving "[name] - Director" in a Production Credits field, along with "[name] - Key grip", etc.
Do you want to start a new thread on this in the MC forums? The timing is correct now...
I think I've already expressed my views in the thread I mentioned. But if you have something to say, I'll probably find it impossible not to add my 2 cents. ;)
I was just suggesting a generic approach ("New Plugin Type") but a simpler solution may work as well.
I'm sure I don't understand all the technical ramifications, but if you can write a MC plugin that accesses the database directly and doesn't need PVD running as a server, that has to simplify the task considerably. On the other hand, if it's simplified too much, it doesn't do much more than the export-import approach. :-\
I imagine this would be something that would necessarily determine all the fields used in a particular PVD database, and facilitate the mapping of any or all those fields to fields in a particular MC database.
What I was leading to with this half-thought is that what most users need help with is the configuration, which is largely a field mapping exercise. Maybe a plugin would be most helpful to the extent it made that as painless as possible. Starting with the premise users are able to configure PVD to get exactly the information they want, imagine this: Your plugin reads their database and sets out all the fields used. For each field, it lists the fields available in MC for mapping, or the option of creating a custom MC field—of the appropriate field type. It would revisit this mapping configuration whenever it detects a field has been added, renamed, changed in type or deleted in PVD. It might even create a default Library View—to arbitrarily display all the fields being populated from the PVD database. I think something like this would be a big help to the "average" user in configuring and maintaining the integration of the two applications.
-
Yes. That's just my way of describing a generic format for all credits. More specifically, it would be "[name] - [role]" for actors, and "[name] - [function, position or job title]" for production credits. It could then be used for any type of credit. For example, you could chose not to have a separate Director field, saving "[name] - Director" in a Production Credits field, along with "[name] - Key grip", etc.
What if you use semicolon delimited lists for: Roles = name1 - role1; name2 - role2; ...
For example:
Credits = Peter Sellers - Actor (German RocketScientist, The President, English Colonel); Raldo - Key grip; ...
You could use their language to pick actors, key grips, etc., directors.
The flexibility of the metadata view would be even as important. There should be a default config, and flexibility for advanced users..
I think I've already expressed my views in the thread I mentioned. But if you have something to say, I'll probably find it impossible not to add my 2 cents. ;)
I'm thinking: Take the essence of the beforementioned thread and start over again in a new thread. That'll "refresh your package" and make it easier for others to get into your train of thought..
What I was leading to with this half-thought is that what most users need help with is the configuration, which is largely a field mapping exercise. Maybe a plugin would be most helpful to the extent it made that as painless as possible. Starting with the premise users are able to configure PVD to get exactly the information they want, imagine this: Your plugin reads their database and sets out all the fields used. For each field, it lists the fields available in MC for mapping, or the option of creating a custom MC field—of the appropriate field type. It would revisit this mapping configuration whenever it detects a field has been added, renamed, changed in type or deleted in PVD. It might even create a default Library View—to arbitrarily display all the fields being populated from the PVD database. I think something like this would be a big help to the "average" user in configuring and maintaining the integration of the two applications.
Yes, one's balancing a thin line between ease of use and flexibility. A plugin with a default, unchangeable map and then additional configurable maps would be the way to go.
-
At some point, If JRiver decide to make a metadataview, you coud use their language (smart lists) to pick actors, key grips, etc., directors
You're right. I'm too used to thinking about from a relational database point-of-view, which MC is not. So an actor is not related to a movie via a role in MC—it can only search selected fields for the text of the actors name. This also means my idea of using one list field for all credits is not a good one. MC would not be able to simple things like list movies of a particular director—you would have to search for "[name] - Director" in Credits, rather than "[name]" in Director.
That'll "refresh your package" and make it easier for others to get into your train of thought.
It would help if I had a train of thought that goes in one direction. ;D
Yes, one's balancing a thin line between ease of use and flexibility.
The main idea behind my view is to use PVD to collect the data in an organized and controlled fashion. Having done that, all that's required is a way to dump it all into MC in some reliable fashion (i.e., data going into the correct field type, no conflicts with existing field use, etc). Once it's there, the user can decide what to display and make use of. I have no problem with the idea of providing default field mappings for the sake of ease of use, but at the same time, I believe the whole thing needs to be fully configurable. Mapping Director onto Director and Genre on to Genre is reasonably safe. Title is pretty obvious, but some will think "the title is the title," while others understand there are titles, original titles and AKA's. And despite what many may think, there is no standard meaning for things like Category and Tag—you can't even say for sure what field type is required. All the potential for confusion is removed if all the database design decisions are made in PVD, and then the plugin preserves that in the transfer to MC.
But I'm getting ahead of myself. I have no idea how difficult it is to create a plugin that might do things like this. Do you think it feasible to make one that lists PVD fields (and their types), offers default mappings to compatible field types and offers to create additional custom fields if necessary? Or were you thinking of something considerably more basic than this?
Open PVD, search folders and export... Done.
If, in the end, you can say the same sort of thing about a plugin, I'm sure you would make many MC users very happy. Most who are interested probably don't have the time or inclination for anything that's not quick and efficient. Once they realize MC is not going to collect the data automatically for them, they'll appreciate the solution: "Configure PVD to get whatever data you want. Install the plugin. Done!"
-
You're right. I'm too used to thinking about from a relational database point-of-view, which MC is not. So an actor is not related to a movie via a role in MC—it can only search selected fields for the text of the actors name. This also means my idea of using one list field for all credits is not a good one. MC would not be able to simple things like list movies of a particular director—you would have to search for "[name] - Director" in Credits, rather than "[name]" in Director.
Yup, that's exactly the point. Datamodelling is *hard* and I think that's one of the reasons why JR probably sticks to their current model. Also, do you think they'd be able to make MC that fast with a relational database?
It would help if I had a train of thought that goes in one direction. ;D
Yes. Doable and concise suggestions may be implemented. Other, vague ideas are hard to implement because there is no "design".
-
Nostra, are you the owner of PVD?
Yes
If yes, would you be interested in cooperating towards making a JRMC plugin where PVD serves JRMC's needs for meta data?
I am interested in everything that can improve PVD user experience. The only problem is that I do not have too much time for thing like this. Rick knew it and so he asked if you could write the plugin.
The MC API looks pretty good indeed and I think the best way for your task is writing a PVD export plugin that uses the API to add/update records in MC. You will also be able to add a configuration window to set up fields easily. Once the plugin is set up the only thing you will need to do to update information in MC will be hitting the Export button in PVD ;)
Also, do you think they'd be able to make MC that fast with a relational database?
I can't tell for sure what they are using, but it is in fact the best way to achieve good performance in managing big datasets is to use a relation database. I think the MC guys are using one as well.
-
I can't tell for sure what they are using, but it is in fact the best way to achieve good performance in managing big datasets is to use a relation database. I think the MC guys are using one as well.
That's the interesting thing. They're not using a relational database. Following is a comment from the lead developer. It's from years ago, but I'm sure he would say the same thing today.
The buzz about using a "relational database" on Interact is mostly misguided. MC uses a system highly-tuned for storing the kind of data that's thrown at it. That same system would allow relational artist and album tables, but this isn't something we think makes sense in the UI for most users.
I think this means they've decided it best it look and behave like a flat file database, but if some part of it needs to be handled as a relation for performance reasons, they will do so. The result is very impressive. It apparently handles very well with databases having 100,000's records.
Raldo, this make me think of something we should probably clarify. I've always wondered why MC developers bother asking what fields they would like to see added to the database, when users can easily add their own custom fields. One reason might be so they can consider the optimal handling of each that is added. So, for example, perhaps the existing Genre and People are handled in a relational way, so they can be used for movie genre and actors. But maybe if custom fields are added for category, actors, director, etc., those would just be handled as regular flat file fields. So the mapping of PVD fields to standard fields provided for the same or similar purpose in MC may be important. I've added a question to your New default video database fields (http://yabb.jriver.com/interact/index.php?topic=51208.msg349897#msg349897) topic at the MC forum.
-
to add a configuration window to set up fields easily. Once the plugin is set up the only thing you will need to do to update information in MC will be hitting the Export button in PVD ;)
Ok, I'll take a look at you plugin examples when I get back from vacation in a weeks time.
In the meantime, could you please point me in the direction to a workable/compilable plugin example, preferably in C++? I know there are som examples in the forum, but it'd be good to start off with the newest version of a workable example...