Author Topic: File Scanner Improvements II  (Read 16999 times)

0 Members and 1 Guest are viewing this topic.

Offline CAD

  • Older Power User
  • *****
  • Posts: 168
  • I've got my eye on you!
    • View Profile
File Scanner Improvements II
« on: August 14, 2010, 11:11:49 am »
Hi,

When scanning movies in from Hard drive, the "media label" is automatically populated with the volume of the hard drive.
I want to use the volume label as a reference to which hard drive a movie is on.
eg. avatar is on hard drive with volume label - movie001, the matrix is on hard drive with volume label - movie002 etc.

Would it be possible to have "scan folders for new movies/changed paths", also update the "Media Label" field with the Volume as part of the change path scan?
three fingered salutation

Offline nostra

  • Administrator
  • *****
  • Posts: 2852
    • View Profile
    • Personal Video Database
Re: File Scanner Improvements II
« Reply #1 on: August 14, 2010, 01:37:47 pm »
Yes, it sounds logical.
Gentlemen, you can’t fight in here! This is the War Room!

Offline CAD

  • Older Power User
  • *****
  • Posts: 168
  • I've got my eye on you!
    • View Profile
Re: File Scanner Improvements II
« Reply #2 on: August 14, 2010, 02:05:47 pm »
Cool,

I'll add it to your TODO list  ;D
three fingered salutation

Offline rick.ca

  • Global Moderator
  • *****
  • Posts: 3241
  • "I'm willing to shoot you!"
    • View Profile
Re: File Scanner Improvements II
« Reply #3 on: August 14, 2010, 08:55:48 pm »
In a situation where a changed path is detected, it's possible the file is not the same as the original. For example, it may be one of a better quality replacing the original. So maybe the improvement should be a processing option to rescan files where the path has changed. That would ensure all the file information is correct. It would have to be optional, as the scanner is also used to update paths when entire folders or collections are moved.

Offline nostra

  • Administrator
  • *****
  • Posts: 2852
    • View Profile
    • Personal Video Database
Re: File Scanner Improvements II
« Reply #4 on: August 15, 2010, 12:48:15 am »
So what exactly are you suggesting, Rick? I think the file Scanner already does what you describe and it will also update the media label in future.
Gentlemen, you can’t fight in here! This is the War Room!

Offline rick.ca

  • Global Moderator
  • *****
  • Posts: 3241
  • "I'm willing to shoot you!"
    • View Profile
Re: File Scanner Improvements II
« Reply #5 on: August 15, 2010, 02:24:07 am »
If I were to obtain a file of a better quality than one I have, put it in a different folder, delete the original file, then run the scanner, it would detect a changed file path and update the path—without rescanning the file. (At least I assume so...Why else would CAD be asking for this?) If it did rescan the file, it would update Media label—but will also ensure all other file information is correct.

It's a minor thing, but from a design point-of-view, I think it's better to update things such as the file information from the same source (the file scanner) at the same time. Exceptions make data inconsistencies more likely. Or to put it another way, when I look at my file information data, I like to know it all came from the last scan of the file. And not that one field might have been changed by Scan folders and another might have been changed by a plugin. If an update is called for, rescan the file.

Offline CAD

  • Older Power User
  • *****
  • Posts: 168
  • I've got my eye on you!
    • View Profile
Re: File Scanner Improvements II
« Reply #6 on: August 15, 2010, 03:09:15 am »
Quote
I think it's better to update things such as the file information from the same source

Rick are you saying, you would like file scanner to update, all fields related to media information when rescanning?
(not going to list all fields) but quality, duration, video codecs, Media label etc.

I think this a good idea. Adds a bit more processing time, but no different than adding a new movie.
three fingered salutation

Offline rick.ca

  • Global Moderator
  • *****
  • Posts: 3241
  • "I'm willing to shoot you!"
    • View Profile
Re: File Scanner Improvements II
« Reply #7 on: August 15, 2010, 03:44:28 am »
Yes. As I said, it should be optional. The default state for me would be "on." If there were a handful of path changes in my scan results, I would not likely notice the file scan time. It takes about 1/2 second per file. If I moved 120 files to a different folder, I might want to turn it off to spare myself that agonizing minute of useless scanning. If I had moved my entire collection to a different drive, it would take a while (and a lot longer than the method you suggested). But for one-time things like that, I don't mind going for a coffee. ;)

Offline CAD

  • Older Power User
  • *****
  • Posts: 168
  • I've got my eye on you!
    • View Profile
Re: File Scanner Improvements II
« Reply #8 on: August 15, 2010, 08:59:20 am »
One more thing, it should scan for path changes AND Volume changes separately.
I have movies stored on drive caddies (removable drive bays.)

Main movie directory is F:\dvd\moviename (year). The volume label of the drive is Movie001.
When this drive is full (and I have watched all movies on it) I will remove the drive and put it into storage (on a shelf most likely).
The new drive I put in will be mapped to F:\dvd\moviename (year) but will have a volume label of Movie002. Scanner folders will need to check for Media label changes as well as path changes to cover all contingencies.
three fingered salutation

buah

  • Guest
Re: File Scanner Improvements II
« Reply #9 on: August 15, 2010, 12:03:43 pm »
Rick are you saying, you would like file scanner to update, all fields related to media information when rescanning?
(not going to list all fields) but quality, duration, video codecs, Media label etc.

Quote from: rick.ca
Yes

If I properly understood discussion, then choosing action "Add New Movie" instead of "Change File Path" from the context menu on a selected title in the File scanner, does exactly what are you suggesting as to be implemented?

I'm sorry if I misunderstood suggestion(s).

Offline CAD

  • Older Power User
  • *****
  • Posts: 168
  • I've got my eye on you!
    • View Profile
Re: File Scanner Improvements II
« Reply #10 on: August 15, 2010, 02:03:18 pm »
Quote
then choosing action "Add New Movie" instead of "Change File Path" from the context menu on a selected title in the File scanner, does exactly what are you suggesting as to be implemented?

Not sure - haven't tried it.
What I'm suggesting is when you scan for HardDrive  path changes (under tools/scan folders for new movies-changed paths), it also checks for changed volumes (the name you give your hard drive). (it doesn't do this at the moment) This is all done locally - no need to download anything. I guess adding a new movie would detect its a duplicate and would download content as well.

hope that clears things up.
three fingered salutation

buah

  • Guest
Re: File Scanner Improvements II
« Reply #11 on: August 15, 2010, 02:35:16 pm »
Try it, don't guess and hesitate, please. Simulate. Rename some other movie on some other hdd different by size of the one existing in your database. Scan that renamed movie. Right click in the scanner on it. Choose the option "Change action to "Add new movie"". Process it.

If it's good, rescan again, now with the original movie to get back the proper database record

I'm not sure what you meant by "duplicate" and "downloading content".

This action "purges" (overwrites) data about location and file information, and adds new data - those you and rick were asking for (media label, video, audio codec, path, all). The only thing is that you get additional screen-shots which you may want to erase afterward (or in a case of a "better rip" previously to erase screenshots of a "bad rip".

Again, maybe I comprehended something incorrectly.
« Last Edit: August 15, 2010, 02:41:21 pm by buah »

buah

  • Guest
Re: File Scanner Improvements II
« Reply #12 on: August 15, 2010, 05:35:24 pm »
P.S. Actually, the only option that doesn't exist at the moment regarding File scanner is the one I asked for: to allow getting an additional database record regardless the existing title as the one in the database is recognized by File scanner

Offline CAD

  • Older Power User
  • *****
  • Posts: 168
  • I've got my eye on you!
    • View Profile
Re: File Scanner Improvements II
« Reply #13 on: August 16, 2010, 04:14:10 am »
Hi buah.

The "scan folders for new movies/changed paths" options seems to be scanning for fields related to the media - a local change.
ie It scans the file(s) sitting on your hard drive for information for information about it.
 
Add movie requires searching /downloading content from internet.

The way I think change path should work is:
Program detects path has changed OR Media Label (Drive Volume)  and updates this information.
(optionally updates other fields related to the file media - such as video codec/audio codec etc).
Itr gets all fo this information from the actual file that is stored on hard drive.

Adding it as a new movie would require (re)downloading content from internet and comparing for changes.

Hope this clears up what i am trying to say.

PS - I will try your suggestions when I get a chance.
PPS - This issue is not related to your original post "File Scanner Improvements" Sorry to hijack the Subject title. :)
 
« Last Edit: August 16, 2010, 05:23:08 am by CAD »
three fingered salutation

Offline rick.ca

  • Global Moderator
  • *****
  • Posts: 3241
  • "I'm willing to shoot you!"
    • View Profile
Re: File Scanner Improvements II
« Reply #14 on: August 16, 2010, 08:10:55 am »
I've made a huge mistake. :'(

But first, the little one...

If I properly understood discussion, then choosing action "Add New Movie" instead of "Change File Path" from the context menu on a selected title in the File scanner, does exactly what are you suggesting as to be implemented?

OIC. Finally. Maybe. I always assumed "Add New Movie" meant "Add New Movie"—not "replace existing file with this file and rescan." ::)

After figuring this out, I decided to experiment further and see if I could improve my scanning routine. My mistake was forgetting I had learned long ago to leave well enough alone. The scanner works fine in a one-movie/one-file situation. A file is scanned and matched to a movie record. If the movie record has no file recorded, it must be a new addition. If the path is not the same, the movie's file must have been moved. But if there are multiple files for one movie, then things become more challenging.

Now, regrettably, I'll have to explain why I have multiple files for some movies. I tend to download, watch and delete. That's not an issue for PVD—it doesn't care if the file in the file path has been deleted. But it is for my media manager. It can only manage information for files that exist. Furthermore, it gets all it's video data from the PVD database, and the common file path is what links each record in the two databases. My workaround is to use a macro that creates a null file with the same name, moves that to a "viewed" folder, and then deletes the file. Both the PVD and media manager scanners detect and record the change, and the meta data for the dummy file is automatically updated. Strange, but all this works fine.

Things get more complicated, however, when I decide to keep a movie after viewing it. In order to have a record of viewing the movie in the media manager, I have to create the dummy file. Unfortunately, the scanner is not smart enough to propose adding the new dummy file to the existing movie file—it will only suggest replacing it. I can override this by changing the action to "Add New Movie." That will add another entry, and "Same movie" can be used to combine them. That, of course, is not at all practical—it's easier to just manually change the file path. Until recently, this has worked fine for me—probably because the number of movies I keep is small.

I recently started collecting trailers. I decided they'd be worthwhile having if they could be recorded in the PVD file path, and thereby integrated with my media manager. That much works very well. I use PVD to find the trailers, and as they're downloaded, I rename them by cutting and pasting the movie file name. At the same time, I cut and paste the trailer path name back into File path (before the movie path, so it would play first). I'll probably continue to do this, rather than rely on the scanner to add the trailers.

What didn't occur to me until I had a sizable collection of trailers, was this would "break" my scanning routine. Now, if I attempt to replace a viewed movie with a dummy file and rescan, the scanner is only able to suggest replacing the trailer (which I want to keep) with the dummy file.

I still haven't got to my huge mistake. Some testing suggested to me (incorrectly, it seems), that the scanner could handle multiple files if they were named in the same way multiple disc movies are. So I renamed all my trailers by appending "-1," movies with a "-2" and view log files with a "-3." Rescanning worked—after fixing lots of exceptions. But it made no difference when it came to moving/changing one of those files. So I tried moving them into the same folder and rescanning. It still didn't work. To add insult to injury, the scanner was unable to properly detect the changes when I moved the files back to their respective folders.

So after many hours of recovery, I'm back where I started. It seems I can't record trailers in File path and use the scanner to detect the change of the media files to dummies as movies are viewed. I'm sure I'm the only one with this particular issue, so maybe I'm expecting to much. But I find it disappointing the scanner does not compare each file detected to the files already associated with a movie, and propose actions based on that. Most of the time, the correct action would be to remove the file reference that no longer exists, and if a file previously referenced still exists, leave it and add the new file.

And as weak nod to staying on topic... If it worked this way, it would be possible to determine whether a file scan is necessary and, if so, how it should be done (i.e., multiple or single file scan). :-\

buah

  • Guest
Re: File Scanner Improvements II
« Reply #15 on: August 16, 2010, 08:55:30 am »
While it was really hard for me to track your routine (:-\), I'll point that there are two options in the context menu for a file(s) scanned in a File scanner window:

1. Add new movie

2. Change action to "Add new movie"


I suggested to use the second option, because that option gives results you've asked for at the beginning of the topic. In addition, if I understood rick's most recent post, there is a checkbox "Add to existing files" in the window that opens after choosing second option, mentioned above. Maybe that could help with adding trailers or any other video file(s)? It works for me in cases when adding "Makin of" documentaries for a movie, for instance.
« Last Edit: August 16, 2010, 08:57:53 am by buah »

Offline nostra

  • Administrator
  • *****
  • Posts: 2852
    • View Profile
    • Personal Video Database
Re: File Scanner Improvements II
« Reply #16 on: August 16, 2010, 10:04:38 am »
Hmm, it seems like the scanner will need a partial rewrite for the version 1 :-/
Gentlemen, you can’t fight in here! This is the War Room!

Offline rick.ca

  • Global Moderator
  • *****
  • Posts: 3241
  • "I'm willing to shoot you!"
    • View Profile
Re: File Scanner Improvements II
« Reply #17 on: August 16, 2010, 11:18:24 am »
Quote
While it was really hard for me to track your routine...

They're mentioned in there somewhere. :P

In the case where the scan has found only one of two files...

1. allows whatever file(s) were missed by the scan to be added as a separate movie, which then has to be combined with the original using "same move;" not only is this a PITA, it's not likely to result in the files being in the correct order (i.e., trailer first).

2. will just result in the one file being recorded, and the file information will be updated.

So, fiddling with the scanner results is not a efficient as simply changing the results directly. And that's a lot safer, as in the ordinary course of things one would have to be very careful to make all the necessary adjustments to the scan results. Curiously, if the existing File path contents are deleted before, the scanner will correctly add all the existing files. That might be the workaround to use in my regular work flow. After viewing a movie, I'll mark it viewed, rate it, delete File path and replace the media file with a log file. Then (maybe at a later time) I'll run the scan.

I'll just have to move all my trailers from \Trailers to something that comes before \Movies. Then the scanner will add the files (including the logs in \Viewed) in the correct order. Unfortunately, because of this issue, I can't just rename the folder and rescan. ::) :'(

Quote
Hmm, it seems like the scanner will need a partial rewrite for the version 1 :-/

You mean it wasn't getting a complete rewrite anyway?! ;D

Maybe you already know what the problem is—I still have difficulty understanding exactly what it's doing. I wonder if there's any promise of an easy fix in my latest observation (highlighted above). If the scanner is able to correctly add all matching files when there's nothing in File path, then it shouldn't be confused by existing data. On the other hand, I don't think it is or should be quite that simple. For example, if a file has been moved, it doesn't seem right that the scanner should fail because of a regex issue. It should be able to match the file to the movie on the basis a file with the same name is currently recorded in that movie's File path. :-\

Offline CAD

  • Older Power User
  • *****
  • Posts: 168
  • I've got my eye on you!
    • View Profile
Re: File Scanner Improvements II
« Reply #18 on: August 16, 2010, 12:36:13 pm »
Quote
Try it, don't guess and hesitate, please. Simulate.
Quote
choosing action "Add New Movie" instead of "Change File Path" from the context menu on a selected title in the File scanner,
Ok tried this. Didn't work.

The scanner needs to search that the  "Volume label" of the drive has changed as part of its detection. Not just the path.
I can have the same path but with a different volume label. ie when I swap drives out.

If I change path and then add movie - Yes it does update Media label (volume), but it also re-downloads everything from internet (not what I wanted and not necessary. This has already been done (and checked to be correct).
Selecting "Same movie" does not update Media label.

I do not get to choose anything if the path has not changed (but volume label has) to select add movie as an option.


three fingered salutation

buah

  • Guest
Re: File Scanner Improvements II
« Reply #19 on: August 16, 2010, 04:32:51 pm »
It works, at least for me. See complete step-by-step process in pictures attached

[attachment deleted by admin]