Welcome to the Invelos forums. Please read the forum rules before posting.

Read access to our public forums is open to everyone. To post messages, a free registration is required.

If you have an Invelos account, sign in to post.

    Invelos Forums->DVD Profiler: Desktop Feature Requests Page: 1... 5 6 7 8 9 ...17  Previous   Next
Importing data from online resources like IMDB
Author Message
DVD Profiler Unlimited RegistrantStar ContributorKulju
Registered: March 14, 2007
Finland Posts: 2,337
Posted:
PM this userView this user's DVD collectionDirect link to this postReply with quote
Quoting surfeur51:
Quote:
If you had read carefully my post, you should have seen that I was speaking of cast and crew.

I read it again and I still cannot find the part where you say that you were only comparing Cast & Crew sections of the dbs.
Quote:
In fact, about the fields you mention, most of them are very quick and simple to fill,

YES, as are all the other fields excluding Cast & Crew and Cover scans. That's why I don't understand why people doesn't contribute. It takes about 5min to create complete profile excluding fields mentioned above. IMDB doesn't offer any info on these release specific fields. After that it's up to you (general) how long you want to use time with those three fields.
Quote:
except overviews  and cover scans. But for both of them, Invelos database is also very poor (decision to reproduce spelling mistakes in overviews, and online scans are rarely what we have in hand).

Overview is accurate and that's the way I like it to be. Do you really think that online scans are rarely what we have in hand? Yes I know that we don't have an option for alternative covers and sometimes online profile contain original release scan and you have re-release. But rarely, seriously? BTW, again, IMDB db doesn't have these fields at all (Yes, they have Storyline, but again, apples and oranges).
 Last edited: by Kulju
DVD Profiler Unlimited RegistrantStar ContributorAddicted2DVD
Registered: March 13, 2007
Reputation: Highest Rating
United States Posts: 17,330
Posted:
PM this userEmail this userView this user's DVD collectionDirect link to this postReply with quote
I have to say... I see it the same way as Kulju.
Pete
DVD Profiler Unlimited RegistrantStar Contributorateo357
Registered: December 27, 2009
Reputation: Highest Rating
United States Posts: 5,131
Posted:
PM this userDirect link to this postReply with quote
Quoting DoubleDownAgain:
Quote:
Quoting ateo357:
Quote:
I started out doing minor corrections to profiles. Once I understood what and how to use the program & CLT, I started doing full cast & crew. Lately I've been doing common name thread corrections in the database (not limited to just my region) making sure that the credited name is correct and adding the common name (Luis Guzman, what a pain. One of the few names I decided to work on.) So sometimes I may be just changing 1 name in a profile and adding any confirmed BY if they are missing. It has slowed down my viewing and correcting of my own DVD's.

I just didn't see a point of having a common name thread with confirmed name credits and no changes to existing profiles being done.


While I appreciate your work, I don't see the point.  I't s like putting a band aid on a gun shot wound.


every little bit helps. Even with gunshot wounds.
DVD Profiler Unlimited RegistrantStar ContributorMithi
Sushi Annihilator
Registered: March 13, 2007
Reputation: Superior Rating
Germany Posts: 2,217
Posted:
PM this userEmail this userVisit this user's homepageView this user's DVD collectionDirect link to this postReply with quote
Quoting JMG51:
Quote:
Why anyone should care, or deny someone else this option is beyond me.

As others already pointed out: It would take Ken time, his most precious commodity, to do this for data we[tm] don't want anyways.
Not to mention that for every structural update of the site that is scraped you'd need to update the importer.

So PluginIn would be the way to go, as already happened for IMDB. And if there isn't anyone interested in writing such a tool you could always use the XML that copy&paste in the DVD Profiler Editor produces to ease the importing.

cya, Mithi
Mithi's little XSLT tinkering - the power of XML --- DVD-Profiler Mini-Wiki
 Last edited: by Mithi
DVD Profiler Desktop and Mobile Registrantmediadogg
Aim high. Ride the wind.
Registered: March 18, 2007
Reputation: Highest Rating
United States Posts: 6,454
Posted:
PM this userVisit this user's homepageDirect link to this postReply with quote
Another plugin example is BulkEdit. It has built-in functions to import CSV, CollectorZ (at least as of a couple of years ago), DVD Spot (obsolete web site) and DVDP XML.

The CSV import can be customized, but it defaults to the format created by the CSV Export plugin. If you have a program that creates DVDP XML from IMDB (or anywhere else), BulkEdit can create a local database from it. But you can't upload it, and if you try to refresh from Invelos, all the IMDB metadata will be overwritten, so it would be a local-only process.
Thanks for your support.
Free Plugins available here.
Advanced plugins available here.
Hey, new product!!! BDPFrog.
 Last edited: by mediadogg
DVD Profiler Desktop and Mobile RegistrantStar ContributorTheMadMartian
Alien with an attitude
Registered: March 13, 2007
Reputation: Highest Rating
United States Posts: 13,201
Posted:
PM this userEmail this userView this user's DVD collectionDirect link to this postReply with quote
Quoting JMG51:
Quote:
Quoting DJ Doena:
Quote:
Quoting JMG51:
Quote:
Why anyone should care, or deny someone else this option is beyond me.


Because it will cost everyone else actual money if Ken did it.


There are (legal) ways around this.

Which are?  Ken does not incorporate a tool to scrape data because IMDb forbids this practice.  If you know a legal way around this, please share.
No dictator, no invader can hold an imprisoned population by force of arms forever.
There is no greater power in the universe than the need for freedom.
Against this power, governments and tyrants and armies cannot stand.
The Centauri learned this lesson once.
We will teach it to them again.
Though it take a thousand years, we will be free.
- Citizen G'Kar
DVD Profiler Unlimited RegistrantStar ContributorCharlieM
Registered Sept 5 2005
Registered: May 20, 2007
Reputation: High Rating
United States Posts: 2,934
Posted:
PM this userVisit this user's homepageView this user's DVD collectionDirect link to this postReply with quote
From IMDB http://www.imdb.com/help/show_article?conditions

Quote:
License and Site Access
Subject to your compliance with these Conditions of Use and your payment of any applicable fees, IMDb or its content providers grants you a limited, non-exclusive, non-transferable, non-sublicenseable license to access and make personal and non-commercial use of the IMDb Services, including digital content available through the IMDb Services, and not to download (other than page caching) or modify this site, or any portion of it, except with express written consent of IMDb. Additional license terms may be found in the Terms. The IMDb Services or any portion of such services may not be reproduced, duplicated, copied, sold, resold, visited, or otherwise exploited for any commercial purpose without express written consent of IMDb. This license does not include any resale or commercial use of any IMDb Service or its contents or any derivative use of this site or its contents. All licenses are non-exclusive and all rights not expressly granted to you in these Conditions of Use or any applicable Terms are reserved and retained by IMDb or its licensors, suppliers, publishers, rightsholders, or other content providers. You will use all IMDb Services in compliance with all applicable laws.

Robots and Screen Scraping: You may not use data mining, robots, screen scraping, or similar data gathering and extraction tools on this site, except with our express written consent as noted below.

Framing: You may not frame or utilize framing techniques to enclose any trademark, logo, or other proprietary information (including images, text, page layout, or form) of IMDb without express written consent.

Meta Tags: You may not use any meta tags or any other "hidden text" utilizing IMDb's name or trademarks without the express written consent of IMDb. Any unauthorized use terminates the permission or license granted by IMDb.

Licensing IMDb Content; Consent to Use Robots and Crawlers: If you are interested in receiving our express written permission to use IMDb content for your non-personal (including commercial) use, please visit our Content Licensing section or contact our Licensing Department. We do allow the limited use of robots and crawlers, such as those from certain search engines, with our express written consent. If you are interested in receiving our express written permission to use robots or crawlers on our site, please contact our Licensing Department.

Linking to IMDb.com: You are granted a limited, revocable, and nonexclusive right to create a hyperlink to IMDb.com so long as the link follows our linking guide and does not portray IMDb, its services in a false, misleading, derogatory, or otherwise offensive matter. You may not use any IMDb logo or other proprietary graphic or trademark as part of the link without express written permission except as outlined in our help section.


And http://www.imdb.com/licensing/

Quote:
Content Licensing

IMDb licenses content to a wide selection of businesses including movie studios, cable companies, websites, video retailers, software developers, electronics manufacturers, mobile applications, and more. We offer content licensing packages that are customized to meet your needs based on your specific audience and the data being licensed. IMDb licensing content includes: cast & crew, user ratings, plot summaries, release dates, box office, keywords, filmography credits, awards, biographies, nicknames and more.


If Ken were to incorporate a scraping feature in to DVD Profiler, he would be violating the License and Site Access section of the agreement. 

Also if he were to incorporate it within the program, which is a commercial product along with it's DB, then he would need to pay for a content license.  If he were to do that, there would be a large reoccurring yearly fee, that he would need to recoup, therefore a yearly fee would need to be incorporated into DVD Profiler(which many would consider a violation of the forever fee that Ken charges now).

I would actually prefer him not to include it, and actually improve the program aspects that we need improved (which has been discussed)

As always My Opinion

Charlie
DVD Profiler Unlimited RegistrantStar ContributorAddicted2DVD
Registered: March 13, 2007
Reputation: Highest Rating
United States Posts: 17,330
Posted:
PM this userEmail this userView this user's DVD collectionDirect link to this postReply with quote
Nice post Charlie... and I agree.
Pete
DVD Profiler Desktop and Mobile RegistrantStar ContributorPantheon
Registered: March 14, 2007
Reputation: High Rating
United Kingdom Posts: 1,819
Posted:
PM this userView this user's DVD collectionDirect link to this postReply with quote
Quoting DoubleDownAgain:
Quote:
But they did, when they forked over the money to Ken.  I think it's going a bit far to call them lazy, as there are other factors involved.  The first and foremost is that it can be somewhat intimidating to contribute.  I didn't contribute for years because of the attitude of some people in the forums.  I'd encourage anyone to contribute , but I don't hold it against those that don't.


First, paying for something does not improve the database. If someone pays for DVDProfiler I assume they realise that it incorporates a USER-BUILT database. So, if the users DON'T build it, it is rendered useless.

I DON'T think it's going too far to call someone who uses other people's data but doesn't contribute any of their own lazy.
All someone has to do is correct a spelling mistake, or add information to one of the countless smaller field and they are contributing useful information.

Many users get hung up on the Cast and Crew and they shouldn't use that as a barrier to contributing.


Quote:
I have much to do and can't be worried about doing a full audit for every submission.  If I'm aware of an issue I'll fix it, but I'm not going to go in depth every time I touch a profile.  I'm happy to share my work with others, but I'm not going to spend my time doing full audits at the expense of getting my own db cleaned up.


Which is EXACTLY what I said is a good thing to do. I NO LONGER feel that a full audit is the way to go. Not everyone has the time or inclination to do so. So ANYTHING they do (as long as it's correct) is of use and a benefit to the database.

Probably best not to quote me stating that I've changed my mind and then proceed to explain why you don't do what I've said you don't need to do.
DVD Profiler Unlimited RegistrantInvisibleman
Registered: March 26, 2007
Posts: 77
Posted:
PM this userView this user's DVD collectionDirect link to this postReply with quote
Quoting Jimmy S:
Quote:
Quoting JMG51:
Quote:
Why anyone should care, or deny someone else this option is beyond me.

Because people will contribute that data to the database and we don't want it neither does Ken (like it or not we don't use anything else than the film credits themselves).


Hello Jimmy S,
JMG51 is totally right, it should be an option. And as far as the uploading is going, you are far from it. Now I think more people are uploading IMDB data to DVD profiler's database. If it would be in option, then this would be controled by the program itself.

As I stated in my 2th post (page1) "And when it is known to DVD profiler, it is easy to add a note (or datafield) in DVD profiler that can not be changed by the user and that is mention the source where the data is comming from. If it knows the data is comming from IMDB, that DVD profiler then also can say, that it will not be uploaded.".

Or do you think that people will download from IMDB first and type all over again o they can contribute to DVD's profiler database? Ones it is controled by the program itself, the data base of DVD profiler would be even cleaner for those, who wish it to be this way.
DVD Profiler Unlimited RegistrantInvisibleman
Registered: March 26, 2007
Posts: 77
Posted:
PM this userView this user's DVD collectionDirect link to this postReply with quote
Quoting DJ Doena:
Quote:
Quoting JMG51:
Quote:
Why anyone should care, or deny someone else this option is beyond me.


Because it will cost everyone else actual money if Ken did it.


As far as the money goes, yes maybe/mostlikely it will cost. But if you would make this optional in paid versions only, then people will start to pay for a full version. So the cost then will be covered, like now people are using and paying for other programs that has this option, but they like DVD profiler as program much better. I only am speaking for a couple of people I know it is for sure and I think many more are out there.

An on the other hand, let's say it will cost $30.000,00 yearly. How many people are using DVD profiler? What will it cost them yearly then $0,50-$1,00? For me, I wouldn't even mind to pay an fee for $10,00-$15,00 if such option was in.
 Last edited: by Invisibleman
DVD Profiler Unlimited RegistrantInvisibleman
Registered: March 26, 2007
Posts: 77
Posted:
PM this userView this user's DVD collectionDirect link to this postReply with quote
Quoting JMG51:
Quote:
Quoting Jimmy S:
Quote:
Quoting JMG51:
Quote:
Why anyone should care, or deny someone else this option is beyond me.

Because people will contribute that data to the database and we don't want it neither does Ken (like it or not we don't use anything else than the film credits themselves).


Hmm, that's not what I get from the op ....

"....Therefore I would like to have an option that make it possible to import the data from an other online source as for example IMDB."

my understanding is he wants the option to be able to import data from an online sourse, as for example, IMDB, to his local profile/machine....NOT to the Invelos database....thus why would anyone care if such an option was to be made available. You use it or you don't.


Yes you hit the nail perfectly. And as I also said, the program knowss, when you are using that option, so the program then also prevent those data to be uploaded to the DVD's profilers database. So in the end you'll have the best of both worlds and everybody is gaining from this, the yes and no sayers, for theDVD's profilers database will stay clean and the others still can use IMDB's data localy.
DVD Profiler Unlimited RegistrantInvisibleman
Registered: March 26, 2007
Posts: 77
Posted:
PM this userView this user's DVD collectionDirect link to this postReply with quote
Quoting Mithi:
Quote:
Quoting JMG51:
Quote:
Why anyone should care, or deny someone else this option is beyond me.

As others already pointed out: It would take Ken time, his most precious commodity, to do this for data we[tm] don't want anyways.
Not to mention that for every structural update of the site that is scraped you'd need to update the importer.

So PluginIn would be the way to go, as already happened for IMDB. And if there isn't anyone interested in writing such a tool you could always use the XML that copy&paste in the DVD Profiler Editor produces to ease the importing.

cya, Mithi


Hello Mithi,

Maybe I'm wrong, but I don't think there is a plugin, only an external program that can do this. Therefore I also asked, that otherwise Invelos, would make it possible to use a lplugin for this. So you can do directly, with plugin. But still then there wil be no control and people will start uploading those data to DVD profilers data base. And that is something you don't want.
DVD Profiler Unlimited RegistrantStar ContributorLewis_Prothero
Strength Through Unity
Registered: May 19, 2007
Reputation: Superior Rating
Germany Posts: 6,730
Posted:
PM this userView this user's DVD collectionDirect link to this postReply with quote
OK,

to make it somewhat simpler.

When buying a Mercedes for having the best comfort, it is absolutely useless to ask the manufacturer for a BMW engine.

Why does anyone think that Invelos should do this?
It all seems so stupid, it makes me want to give up!
But why should I give up, when it all seems so stupid?


Registrant since 05/22/2003
DVD Profiler Unlimited RegistrantStar ContributorAddicted2DVD
Registered: March 13, 2007
Reputation: Highest Rating
United States Posts: 17,330
Posted:
PM this userEmail this userView this user's DVD collectionDirect link to this postReply with quote
I don't see Ken doing this for this specific reason

From imdb as quoted above:
Quote:
Robots and Screen Scraping: You may not use data mining, robots, screen scraping, or similar data gathering and extraction tools on this site, except with our express written consent as noted below.


Since Ken don't want to use them for our database (thankfully!)... and with the posts from Ken that I already quoted... I don't see Ken doing this. As it would go against imdb's terms of usage.

I honestly think if you want something like this you would need to use the stand alone programs offered or a pluggin (if there is one... or if one is ever made).
Pete
DVD Profiler Unlimited RegistrantStar ContributorKathy
Registered: May 29, 2007
Reputation: Highest Rating
United States Posts: 3,475
Posted:
PM this userEmail this userView this user's DVD collectionDirect link to this postReply with quote
Invisibleman,

You did see the posts pointing you to plugins that will do exactly what you want, didn't you?

It seems to me that, although you can import your data from IMDb using one of them, this isn't good enough.

It seems that you want invelos, and only invelos, to implement your idea. Why?

This issue has been brought up countless times throughout the years. And, the owner of the program has never wavered in his stance - a single third party source such as IMDb is not to be used.

Other community members agree with your stance though and have developed plugins that accomplish that goal.

These users spent a lot of their time developing these programs and did so free of charge. But, this data is NOT to be uploaded to the online database because of all the reasons stated throughout this thread.

I suggest that you take a look at those plugins.
 Last edited: by Kathy
    Invelos Forums->DVD Profiler: Desktop Feature Requests Page: 1... 5 6 7 8 9 ...17  Previous   Next