Trying to import an additional 300,000 plus photos into my Lightroom catalog. Hoping Lightroom's up for the task and wondering if the import job will be done by the time I get home from work today.
Jeremy, that's how I've been using it so far, as my workflow tool and not so much for my entire archive and that's still how I'm using it on my Mac. But I thought I'd try importing my 2002-2007 archive on my PC version to see how it handles performance with that large a library. I've been wanting to go back through my archive and do more processing and hopefully it handles the archive library well enough.
- Thomas Hawk
Having one giant archive is probably not a good idea. I maintain archive catalogs broken up by year. I have separate catalogs for unprocessed pictures/projects which I then merge into the archive catalogs as they are completed. I will say that I am considering going back and reprocessing some pictures from a number of years ago.
- jho
the huge archive is more of an experiment. I'm hoping it works because I've still got tons of old unprocessed shots in my archive that I'd like to process.
- Thomas Hawk
well Lightroom's not handling the job so well. When I got home it showed that it was still in the process of importing photos but Lightroom seemed frozen. it was stuck on about 106,000 photos for the upgrade. I canceled the import and went back and am trying to manually synch the individual subfolders but LR is very unresponsive.
- Thomas Hawk
Yikes, Thomas! I'm still trying to get used to LR. I read on a photography forum, for batch imports PS is better... Good luck and thanks for the updates. :)
- Mona Nomura
Are you running on Mac or Windows? If on Mac have you turned on 64-bit mode? Wonder if that will help ... I haven't had too much experience in Lightroom library of that size, however after I turned on 64 bit mode I notice Lightroom is just a tad faster. (I am on MacPro with 8GB RAM)
- Clarence Chiang
OMG that is going to totally tank their prod servers!! ha, ha!! you are a BAD end user!! LOL
- Susan Beebe
I'm running this version on a Windows Vista PC. I'm only using LR2 on my Mac for my production of my current photos. Thought I'd try my copy on the PC for my large archive as I want to go back and process some photos that I skipped the first time around. It's just really really sloowwwww now that I've got over 100,000 RAW files imported into it. Will keep trying though.
- Thomas Hawk
I couldn't even get LR2 to load up after about 5 minutes. I let it sit and am now back downstairs on the Mac for a while. I'll go back and check it in 15 minutes or so and see if it's finally booted up and then try to import even more photos into it.
- Thomas Hawk
Yikes! What percentage do you think you will discard, and how many will go through post-processing? Given how much and often you shoot, I've wondered how you control the volume - not unlike Robert Scoble's social "noise".
- JCunwired
Yes LR import can be quite slow. Although I usually perform a NEF to DNG conversion along with the import. Last week I imported 3000+ shots (from my Greece trip) to LR 2.0 and it took 2+ hours.
- Clarence Chiang
well, I won't discard anything. I never delete photos. Instead I'll just go through the archive and flag a bunch of old ones to process. Not sure how many I'll find that I'll feel like processing. I'm still 2 months behind on my current photo workflow so this will likely be a long term work in progress. I almost always find photos that I passed on the first time around though when going through my archives that I reconsider and process.
- Thomas Hawk
300,000 photos? holy, that would totally crash my browser.
- wiredgnome
TH, if you do a search for Lightroom on my site, you'll see I had similar problems with it. Granted, it was version 1, but anything over 10,000 seemed to cause LR to crash and corrupt the catalog files. That meant having to re-import photos, rebuild previews and thumbnails, and losing meta-data that hadn't been written to the XMP files. LR2 seems more stable, but I've divvied my library by years and keep a catalog for each. It's safer that way.
- Raoul Pop
TH - I read somewhere that there is a limit on the LR catalogs...and I KNOW it was no where near 300,000...not even 100,000. I don't think LR was built to handle LARGE catalogs...or that's what I'm learning at least.
- Justin Korn
There is a known bug that it causes problems if you have too many top level folders. The suggested fix is to add a parent folder.
- jho
it seems to be working much better after letting it sit last night. total photos in catalog at this point 167,640. Going to work this weekend and try to get even more in there. It would rock if I could get all 300,000+ photos in my 2002-2007 archive in there.
- Thomas Hawk
My entire library is 76GB and 26,400 files. No problems here. I am going back to look for needles in the hay stack. Most of those are within the last 12 months though.
- Russellreno
sounds like an instance where defragging after you are done would really help.
- mjc
I love lightroom, but I still haven't figured out how to use it for long term archival; I too use it for current workflow only. One of the reasons it's not very useful for me for long-term archival is that it's so slow at moving files around. Another reason is that for DNG vs JPG, it works exactly the opposite from me - I like the JPGs at the top level "event" folder and the original DNGs in a sub-folder (./originals), while Lightroom likes it the other way around...
- Yaniv Golan
my suggestion yaniv is that you split catalogues out. Probably by year
- Phill Price
Ok, I left Lightroom to run all night and came back this a.m. with an error message that Photoshop (not Lightroom which is odd) had closed. I relaunched it and I've got 198,344 photos imported into it now. Getting closer. performance is definitely an issue though and yesterday was challenging at times trying to use it to process photos.
- Thomas Hawk
"The line snaked around more than two city blocks in Brooklyn. Eager fans waited anxiously, hoping for admission into a prized event on an early June evening. They weren’t lined up to see Miley Cyrus perform or to catch a glimpse of Patrick Dempsey. Instead, they were waiting for Kevin Rose and Alex Albrecht, the stars of one of the most popular Internet TV shows, “Diggnation,” who were slated to do their show live that night from the club Studio B in Brooklyn."
- Robert Seidman
The added perspective of seeing them side-by-side is nice. I wonder where the moon is in it's orbit (does this accurately show the distance between the two)?
- Paul Buchheit
Given the description of Earth's distance from Mars, I'd say this is head-on, meaning this is to scale: "At the time the image was taken, Earth was 142 million kilometers (88 million miles) from Mars, giving the HiRISE image a scale of 142 kilometers (88 miles) per pixel, an Earth diameter of about 90 pixels and a moon diameter of 24 pixels."
- Mark Trapp
I take that back, that would make the moon way too close to the Earth.
- Mark Trapp
Like Robert Seidman, I am more obscessed with Friendfeed that I have been with any site in years. I see it as the next big thing online. - http://www.youhavetobekiddingme.com/2008...
With some tweakings like : threads, and data portability stuff, it would be THE big thing :)
- directeur
It's certainly more attractive to me than Twitter ever was.
- Tom Mulrooney
Same here Tom. Directuer -- you have to give this ~5 years or so to develop. It's different than Facebook or MySpace and if successful the adoption curve will be very different. Steve R.,...thanks!
- Robert Seidman
Yeah, it feeds well through feedalizr and other front end apps and services. But I think it might run into the same problem as Twitter (no, not the scaling problems) - how do you monetise a communication platform without charging a subscription fee?...via feedalizr
- Shannon Low
Robert: I hope they'll never look like Facebook or worse: MySpace! :) But things like data potability are really worth it! And if _simple_ people like me can't promote it, "famous" ones should... Hear me the Scoble? Hear me? :)
- directeur
@Seidman I was reading your newsletter religiously circa 1994. I miss those days.
- Steve Rubel
see all that white space ---------> potentially lots of contextual advertising opportunity. I figure it won't matter. Microsoft will throw $2 billion at them anyway ;-)
- Robert Seidman
The next big thing? I'm the only one in my family with a freakin' FaceBook and MySpace account... HELLO!?
- Ⓒⓗⓡⓘⓢ Ⓟⓘⓡⓘⓛⓛⓞ
l0ckergn0me : Sure, you didn't invented the Internet! I did! :)
- directeur
Directeur, data portability as you think of it may matter a great deal, and someday it may matter a lot to FriendFeed. Today is not that day, nor should it be in this stage of its development.
- Robert Seidman
@Seidman That whitespace is only useful if friendfeed is a destination site, but not if people are accessing it through other services and front end apps. Maybe most people will access it as a destination site. If it becomes a communications backend, I dunno....via feedalizr
- Shannon Low
Robert Seidman: Agree, but why shouldn't it in _this_ stage of its dev?
- directeur
Chris P: I thought AOL was the next big thing in 1985. It took about 10 years. Patience!
- Robert Seidman
Chris, he means the next big thing since AOL.
- Phil Glockner
@directeur, what exactly do you want to port out of friendfeed at the moment?
- Trent Olson
@Shannon, business model is definitely the $64 billion question. My guess is they want it to be a destination, not a backend and that currently most people will prefer web interface to Twhirl, Feedalizr, etc -- but it's just a guess.
- Robert Seidman
Trent: My Likes for example. I can make an APML file from del.icio.us why not FriendFeed? I'd like to import an OPML here, they have the technology and the "power" to support it. Why not OpenID, also?... :)
- directeur
@Steve Rubel: I miss those days too sometimes though I really like how things have progressed. Broadband, tons of free content, iPhones, etc. And J.Phil -- besides the web and all the free content, FriendFeed is the most interesting thing I've seen.
- Robert Seidman
@Seidman Yeah, that's a reasonable guess. I wonder how many people notice the advertising on their webmail screens though. In the end, you're right, Microsoft might just buy them for branding purposes, like they bought hotmail....via feedalizr
- Shannon Low
@directeur: ok, say you export your likes. What would you import them into? To me, data portability only makes sense if there's a logical destination to take that data into. I'm sure that future versions of the api will allow for this type of thing (but since it doesn't allow hiding, joining/unjoining rooms, etc, it may be a while, since these functions are much closer to the core functionality of ff at the moment).
- Trent Olson
@Trent: here: http://feedego.com for example (disc. I made it) I think that my behavior, not only my identity is a part of me, and I should have the ability to port it and use it anywhere... Sure, it's their right to tell me : this is our service, use it as it is or f** off! But I think that services like FF should show the road to others
- directeur
“The world doesn’t need another generic social network. The world needs an infinite number of hyper-focused social networks that are about specific subjects,” Anderson said.
- Noah Carter
Yes and no. For example, the scientific community could/is using social networks to collaborate, share ideas, but you need to be part of a greater community as well. Although I'd argue that limiting yourself to social networking is a very narrow view of the web
- Deepak Singh
FF is the most generic social network - they re missing social network effects of new web 2.0 jobs like FF!
- Erhan Erdoğan
What's missing from the article is noting that advances in computer power can help reduce some of the noise for us, while proving more relevant info recommendations for the masses.
- Mike Reynolds
My 2 cts: First the Internet and the web are 2 different things. Second, that notion of "trendsetters" is just complete bullshit... People still use IRC (which is not web) and no one is a "trendsetter" on IRC. This is plain wrong that people think that social networking was born with social websites. And again, I think that the web is not about trends, but needs.
- directeur
The Web itself should be the social network
- Greg Beck