ACDSee Forums Topic: Manage mode / Batch Set

ACDSee Forums Topic: Manage mode / Batch Set - very slow http://community.acdsee.com/forums/ ACDSee Forums Topic: Manage mode / Batch Set - very slow en Sun, 14 Mar 2010 17:05:19 +0000 Marc Sabatella on "Manage mode / Batch Set - very slow" http://community.acdsee.com/forums/topic/manage-mode-batch-set-slow#post-40022 Thu, 30 Jul 2009 03:53:45 +0000 Marc Sabatella 40022@http://community.acdsee.com/forums/ <blockquote><cite><a href="#post-39966">John Radcliffe said:</a></cite> Where are you seeing reported how many images have been written to? I can't recall ever having seen this. </blockquote> Good question.  Don't know where I got the idea there was count present.  I think when I was feeling my way through all this original, I probably just eyeballed it based on how long it took - if the progressi indicator blows right by a file, I know it didn't write, but if it takes several seconds, I know it's doing something.  That and/or I've looked at the file-modified dates.  Anyhow, it was easy enough to tell that writing Keywords forced it to write everything, but without that, it could be very quick and skip the fiule that hadn't changed.  I suppose you could use this same idea to get a handle on what's going on in your case. John Radcliffe on "Manage mode / Batch Set - very slow" http://community.acdsee.com/forums/topic/manage-mode-batch-set-slow#post-39966 Tue, 28 Jul 2009 14:13:56 +0000 John Radcliffe 39966@http://community.acdsee.com/forums/ One of the reasons I don't use the IPTC Keywords and Categories fields is the discovery that ACDSee always writes to these, even if the data it is writing is identical to what is already there. I keep to fields that ACDSee only writes to if the data has changed, so that it only updates (and I only have to back up) image files that needed to be changed, not every one in the batch. (As I'm the only person who accesses this info I don't find my non-standard usage an issue.) Of course when writing to the image files it's easy to see which files (if any) have been updated by checking the modified dates, but it's not so easy to see what ACDSee is doing when reading back to the database. However, judging by the time taken and by how the progress bar slows down or speeds up, it would seem that ACDSee s doing more work when more data has changed. (Even though I find v2.5 and v3 take longer to write to the image files, I have checked, and they aren't in fact updating more files than v2 did.) Where are you seeing reported how many images have been written to? I can't recall ever having seen this. Marc Sabatella on "Manage mode / Batch Set - very slow" http://community.acdsee.com/forums/topic/manage-mode-batch-set-slow#post-39934 Mon, 27 Jul 2009 19:07:01 +0000 Marc Sabatella 39934@http://community.acdsee.com/forums/ <blockquote><cite><a href="#post-39828">John Radcliffe said:</a></cite> What I may NOT have made completely clear is that my "worst case" scenario involves processing a lot of image files where NO data actually needs to be copied back from the files to the database. In this case v2 can complete 1000 files in under a second, whereas v3 can take 180 seconds. </blockquote> Ah, that sounds like good info, and might explain why I hadn't noticed a big change with respect to JPEG.  Another thing to consider that I remember from testing this on previous versions - if you write certain fields like Keywords, it seems Batch Set Information (BSI) will write all files regardless of whether anything has changed.  If the problem fields like Keywords are *not* part of your template, then BSI can be a lot smarter about only updating files as necessary.  So do be sure the templates you are comparing both include Keywords (or both don't include them) to be sure you are comparing fairly.  Either way, BSI should be telling you how many files were updated - is the difference in speed is because in one version nothing is being udpated and in the other everything is, or just that updating nothing takes longer in 2.5 and 3.0? In any case, once again, I want to emphasize that what I am seeing is, as far as I know, something rather different, since I am shooting RAW, and am definitely talking about cases where everything is updated.  With RAW (proprietary - PEF - as opposed to DNG), the info is written to a small sidecar text file and *should* be incredibly fast.  That is how I remember it being with 2.0, and the Embed Database Information (EDI) facility of 2.5 and 3.0 is still that fast.  As in, hundreds of files in a matter of seconds.  But BSI, which in theory is doing no more work than EDI, is as I keep saying at least 10, probably more like 100 times slower than EDI for these proprietary RAW files - several *seconds per file* as opposed to dozens of *files per second*.  I can see no reason for that. John Radcliffe on "Manage mode / Batch Set - very slow" http://community.acdsee.com/forums/topic/manage-mode-batch-set-slow#post-39828 Fri, 24 Jul 2009 13:33:09 +0000 John Radcliffe 39828@http://community.acdsee.com/forums/ Hi Melanie, The reason I provided a link to this thread was that I thought this was the best way of providing as much information AND background as possible. From Ben's response (which I did acknowledge) it would appear that the information in this thread had indeed been read and taken into account. What I may NOT have made completely clear is that my "worst case" scenario involves processing a lot of image files where NO data actually needs to be copied back from the files to the database. In this case v2 can complete 1000 files in under a second, whereas v3 can take 180 seconds. It is obviously doing much more than just checking that the data IS the same: I just wonder whether really it needs to do this for the images for which no change is necessary (or whether it could do it optionally). The fact that I frequently run Batch Set in this way on entire folders, just to ensure that the data for a handful of images is updated, can be put down to my laziness. It's just so much easier to do that than to have to sort by Modified date, select the most recent files, run Batch Set on just those files, and then re-sort by Filename. So I blame the impressive speed of v2 for encouraging me to be lazy! It's an ease-of-use feature that I'd hate to lose. Marc Sabatella on "Manage mode / Batch Set - very slow" http://community.acdsee.com/forums/topic/manage-mode-batch-set-slow#post-39815 Fri, 24 Jul 2009 00:09:45 +0000 Marc Sabatella 39815@http://community.acdsee.com/forums/ Just to be clear, it wasn't my intent to criticize support.  As some here may know, I used to be a software engineer, and in my world, we considered it a *good* thing when support was able to answer questions without consulting us.  And given the number of engineers who may have worked on the same section of code for different releases over the years, it was by no means a given that any one person would be able to provide the full history of any given feature from release to release.  So I didn't mean it as a bad thing when I suggested support may have answered without having the fully story direct from the source - in my experience, it's practically inevitable that this will happen at times. Anyhow, this answer *does* seem basically sound to me - it just seems like it might not tell the whole story.  I've never measured the difference between 2.0 and later versions for JPEG, so I can't comment on how "slight" it might be - whether John's measurements are abnormal or not.  But for RAW, I can absolutely verify that IPTC write times have slowed down *enormously* - the same kind of factor of ten differences John refers to.  If ACD needs help reproducing this, I'm happy to do what I can. As a point of reference, the embed database operation runs in a fraction of a second per image, and hundred of images can be processed in just seconds.  That's the kind of speed that *used* to happen for propreitary RAW files when writing to IPTC, but now these files are as slow as DNG  which is to say, around 10 times slower.  So I do hope someone looks into that asect of the issue rather than considering the matter closed. Melanie Wood on "Manage mode / Batch Set - very slow" http://community.acdsee.com/forums/topic/manage-mode-batch-set-slow#post-39813 Thu, 23 Jul 2009 23:12:00 +0000 Melanie Wood 39813@http://community.acdsee.com/forums/ Hello, While I recognize the times you have provided are longer, the durations we have been able to reproduce are not as long. The statement, based on our testing, is true. ACDSee Pro 2 was slightly quicker than Pro 2.5 and Pro 3 but it was not performing all the same functions. I would recommend replying to the ticket, ensuring you provide CS with all pertinent information, not just a link to a forum thread. When responding to users questions, CS endeavors to give the highest quality of service. There is consultation prior to responding to new issues and revision of old responses on a regular basis. -Melanie John Radcliffe on "Manage mode / Batch Set - very slow" http://community.acdsee.com/forums/topic/manage-mode-batch-set-slow#post-39785 Thu, 23 Jul 2009 14:05:56 +0000 John Radcliffe 39785@http://community.acdsee.com/forums/ Well I sure hope you're right. In some cases I'm getting times of 2 or 3 seconds for v2 and over 3 minutes for v3 (with v2.5 near to the latter), which simply ridiculous. In retrospect I wondered why I hadn't noticed this slowdown before, as I've been using v2.5 for some time. Then it occurred that it's because 90% of the time I'm reading the metadata back (the case that for me shows the biggest slowdown) on the computer I'm using at the moment. Because that's running Win2k, it still has Pro v2 on it. Marc Sabatella on "Manage mode / Batch Set - very slow" http://community.acdsee.com/forums/topic/manage-mode-batch-set-slow#post-39733 Wed, 22 Jul 2009 20:48:37 +0000 Marc Sabatella 39733@http://community.acdsee.com/forums/ That would be disappointing.  But I've got an alternate read of the situation, not based on any inside knowledge of this particular issue or ACD, but of the software world in general.  My guess is that the response you received was from someone in support who may or may not have had any contact whatsoever with a developer before responding, and even if it was a developer, it might not be one affiliated with this particular tool, or even the Pro 3 project.  I'd further guess that the people responsbile for this tool on pro 3 *will* see this thread and, independently of what someone in support said about 2.5, may well look into it.  I guess we'll see.  I too am concerned about this slowdown, which I agree is not "slight" at all, but a factor of 10 or more (that's what, *1000%*?) when it comes to RAW files. John Radcliffe on "Manage mode / Batch Set - very slow" http://community.acdsee.com/forums/topic/manage-mode-batch-set-slow#post-39585 Fri, 17 Jul 2009 15:29:56 +0000 John Radcliffe 39585@http://community.acdsee.com/forums/ Well, I've had a response from Support, although I have to say that I find it disappointing. <blockquote> While ACDSee Pro 2 was slightly quicker than Pro 2.5 and Pro 3, Pro 2 was not performing all the functions that later versions are. This more complete processing provides users with greater stability but does result in fractionally longer wait time. </blockquote> Now I've no idea what else what else is being done (and how this is of benefit), but I have to disagree with the idea that the results I note above involve a "fractionally longer wait time". So, if the developers consider such an increase to be reasonable, I guess I can't expect a return to v2 levels of performance any time soon -- unless I go back to using v2 for image management and viewing, and either forget v3 completely or just use it for editing. John Radcliffe on "Manage mode / Batch Set - very slow" http://community.acdsee.com/forums/topic/manage-mode-batch-set-slow#post-39443 Mon, 13 Jul 2009 13:06:23 +0000 John Radcliffe 39443@http://community.acdsee.com/forums/ Well, it's now 9 days since I submitted a support ticket (for v2.5, but with a link to this thread). Since receiving the auto-response acknowledging receipt I've heard nothing. So I guess either the developers aren't worried by a 20 times slowdown, or they know what the problem is but don't want to admit it. Personally, I don't find either approach particularly helpful (or, indeed, courteous). Is the slowdown significant? To me it is. With v2 I could simply select all the images in a folder and run Batch Set to copy my data back from the image fields. If only a few needed updating then, even if the folder contained around 1000 image files, that process might only take a 2 or 3 seconds. However, with v2.5 or v3 Beta I'm now faced with perhaps a minute's wait (or I have to just select the specific images first, if I know which they are). John Radcliffe on "Manage mode / Batch Set - very slow" http://community.acdsee.com/forums/topic/manage-mode-batch-set-slow#post-39101 Thu, 02 Jul 2009 12:56:48 +0000 John Radcliffe 39101@http://community.acdsee.com/forums/ Sorry, Marc, I should have made that clear. My files are all Jpgs (so the data is being written into the image file itself). I was hoping that we would have got some response from the Development Team, as this is not the sort of change I expect (or want!) to see between versions, unless there is a very good reason that we are overlooking. As this issue also affects v2.5, I have submitted a support ticket for v2.5. Marc Sabatella on "Manage mode / Batch Set - very slow" http://community.acdsee.com/forums/topic/manage-mode-batch-set-slow#post-39084 Wed, 01 Jul 2009 17:41:12 +0000 Marc Sabatella 39084@http://community.acdsee.com/forums/ Are those for RAW?  I am seeing literally a 10-100X slowdown in IPTC write time from 2 to 2.5 (and 3.0 is as bad as 2.5), but only for RAW.  That seems to be something different than the "slight" slowdown you are seeing. John Radcliffe on "Manage mode / Batch Set - very slow" http://community.acdsee.com/forums/topic/manage-mode-batch-set-slow#post-38825 Wed, 24 Jun 2009 13:11:03 +0000 John Radcliffe 38825@http://community.acdsee.com/forums/ Here are some comparative write times (i.e. when copying data from the database into IPTC fields in the image files). Please note that these are for only 50 images, not the full 350 in the previous exercise, so it's not actually writing quicker than reading. Pro2 = 25sec / 0sec Pro2.5 = 25sec / 2sec Pro3 = 35sec / 3sec Here v2.5 is just as quick as v2, except where no updating is required. Beta v3 takes about 40% longer than v2.5 in each case. These figures aren't exactly impressive, so would anyone from the Development team care to comment? It may be that I'm missing some factor out of the equation, and that the later versions aren't just dragging their feet. John Radcliffe on "Manage mode / Batch Set - very slow" http://community.acdsee.com/forums/topic/manage-mode-batch-set-slow#post-38787 Tue, 23 Jun 2009 13:40:54 +0000 John Radcliffe 38787@http://community.acdsee.com/forums/ Thanks Marc, I thought later that I should have Optimized the database first, so I repeated the test for v3Beta after doing so. It considerably improved the time, down to 35sec from 55! However, Optimized or not, v2 just flies in comparison. I haven't compared write times yet (database to IPTC fields), but I guess I should! Marc Sabatella on "Manage mode / Batch Set - very slow" http://community.acdsee.com/forums/topic/manage-mode-batch-set-slow#post-38744 Mon, 22 Jun 2009 18:14:14 +0000 Marc Sabatella 38744@http://community.acdsee.com/forums/ I hadn't noticed that, but you're right, and those are excellent guesses as to what might have triggered this, although I wouldn't think any of those features *should* have caused this.  FWIW, though, I *have* noticed a possibly related thing: batch set from db *to* a set of RAW files also got much slower with 2.5 for RAW files (and remains so for 3.0).  Odd, since all it is doing is writing a small text file (XMP).  Similar numbers to what you're seeing - 10-100 times slower. FWIW, the embed operation is as fast as batch set used to be, so I know it's *possible* to write those XMP files quickly. John Radcliffe on "Manage mode / Batch Set - very slow" http://community.acdsee.com/forums/topic/manage-mode-batch-set-slow#post-38734 Mon, 22 Jun 2009 15:12:41 +0000 John Radcliffe 38734@http://community.acdsee.com/forums/ Why is Batch Set so slow when copying metadata back from the image files to the database? I hadn't realised that this vast speed decrease had occurred with v2.5 compared with v2, so I tested on a folder. In each case I give two timings. For A I cleared all the Captions and Notes fields and then ran a template in Batch Set to copy them back. For B I just ran the same template again, but this time all it should have to do is check that the contents are the same. Pro2 .........A= 2sec / B= under 1sec Pro2.5 ..... A= 18sec / B= 18sec Pro3 ........ A= 55sec / B= 20sec These are massive increases, so what on earth are v2.5 and v3Beta doing that takes all that time? Or put another way, what changed between v2 and v2.5? I can think of 3 changes but can't understand why any of them should have had an impact on this exercise: (1) The ability to embed ACDSee data in the image files. I'm not using this, and the image files appeared to be unchanged afterwards, so I don't see why this should be relevant. (2) The "Originals" feature, but that should have no relevance either. (3) Unicode support for data fields, but I fail to see why this should slow things down as much as this (if at all). Does anyone have a suggestion? Does someone from the Development team have the answer? Is there anything I can do to speed it up?