CS2/Bridge RAW

[flyfishertoo]

I have a 10D and have been shooting RAW. I took some pictures yesterday. All of them can be view on the camera display. When I downloaded them and try to view them using Adobe Bridge, a few if the images wouldn't preview or open in CameraRaw. I used Adobe DNG converter and it recognized them and converted them to DNG. I was then able to view them in Bridge and open them in CameraRaw.

Have you ever experience this? Do you know what the problem might be?

[cox]

I haven't had that problem except when I copy new files into a directory that I have already used Bridge to browse. Then the thumbs don't update, and the image doesn't show up (I use filmstrip mode), but CS2 can still open them. I haven't found a workaround yet because it's a rare problem for me.

[skagitswimmer]

I started having a similar problem a couple weeks ago - and posted messages here about it.
My thumbnails in Bridge were corrupted to look like this
http://www.dphoto.us/forumphotos/data/500/corrupt_thumbcopy.gif
and I got messages that CS2 could not "parse" the file

I think I have traced the problem to reading and writing from and to my secondary disk. I have my main disk (c: drive) with CS2 on it, and a very large second hard drive which I was using to store my photos. I think trying to edit across the 2 discs was causing problems but I am not sure. HOwever when I copied the directory with the files I was working on to my c: drive things seemed to work better.

[flyfishertoo]

I don't have a secondary disk, and I don't get a "corrupted" image like yours, I get nothing. The file is listed, but no thumbnail and I can't open the file.

[skagitswimmer]

hmm. have you defragged the drive recently? I am just wondering if the issues you and I are having are related.

[jliechty]

Defragmenting your hard drive is worthless to solve a problem like this, and could make things worse! Back things up immediately, and try doing a scan of the disk (go to its properties, select the tools tab, and click Error checking; ensure to search for bad sectors). A potentially failing hard disk is never something to mess around with.

[skagitswimmer]

That's a good comment - you wouldn't expect the problem to be cured for existing photos on a hard drive by defragging. However I would think that defragging the drive would improve its reliability with respect to future photos.

In my case, I updated the firmware with the new releases over the weekend that are said to improve the reliability of transfers from the camera. When I uploaded the next batch from the camera after that, I was prompted to re-install one of the files from the CANON CD-rom that came with the camera. I did. The result seems to be that my raw files transferred flawlessly for the first time in a while.

[MatsP]

That's a good comment - you wouldn't expect the problem to be cured for existing photos on a hard drive by defragging. However I would think that defragging the drive would improve its reliability with respect to future photos.

In my case, I updated the firmware with the new releases over the weekend that are said to improve the reliability of transfers from the camera. When I uploaded the next batch from the camera after that, I was prompted to re-install one of the files from the CANON CD-rom that came with the camera. I did. The result seems to be that my raw files transferred flawlessly for the first time in a while.

Defragging the hard-drive is similar to shuffling the contents of your freezer around to create "more space". In the case of the hard-disk, you don't get more space [1]. It also moves parts of files around so that they are (more)contiguous, which helps the file-system load the files quicker because it doesn't have to move the read/write head around so much.

However, if your compressor on the freezer is beginning to fail, re-organizing the contents will not help much. [Although I suppose one could argue that the heat-loss through a compact lump of frozen goods is less than that of several smaller lumps of frozen goods, but that's a side-effect, rather than the intended idea of re-organizing the fridge, I would say].

Checking the files on the drive is definitely a better approach. Doing backup first is proper care and attention in this case, as the individual files on the disk may well get "more damaged" in the process of repairing any faults on the disk.


[1] You obviously also don't get any more space in the freezer unless you throw things away. But the space is used more effectively making you THINK that there is more space.

--
Mats

[skagitswimmer]

Defragging the hard-drive is similar to shuffling the contents of your freezer around to create "more space".

My understanding is that defragging the drive does'nt create significantly more space but does speed things up by minimizing the time the computer spends searching the hard drive for data. (trying to find both packets of frozen chicken in the freezer).

In the case of computer files, my understanding is that there is less likely to be errors accessing the various packets of information (or leftovers in the freezer) if the files are cotiguous rather than broken up and widely separated.

Of course one should also run a checkdisk or similar utility to block off bad spots (not sure what the freezer analogy would be there).

Checkdisk can (with appropriate options) recover some lost data which is usually then stored in a XXX.chk file in the root of the drive. However I don't think there is any easy way of using that data to fix a corrupted digital photo. It is useful for recovering txt files since you may get the majority of what was lost.

Neither Checkdisk nor defrag will help much in fixing corrupt photo files, methinks. Both are somewhat risky in that you are right - defrag might result in errors in relocating data (that can be eliminated or minimized if you have the data checking feature enabled) and checkdisk can block off a sector that had some accessable data in it. The answer is to keep backing important files up before doing this stuff.

Mats, your knowledge in this area may well exceed mine - I never could find that missing package of chicken in my freezer. I'm just hoping that I have my photo transfer problem solved.

[jliechty]

You're mostly right - except that defragging never actually fixes anything. If your computer is having trouble accessing files, it will be for some other reason - not that it supposedly encounters more potential for error when accessing a file broken into three pieces than a similar file that is contiguous. Remember that this is digital - an exact science - rather than analog. Unless the Master File Table or File Allocation Table gets corrupted due to a problem with the disk itself, a file could be "broken" into a thousand fragments, and the computer would never have any trouble piecing it back together exactly as it should be (although it might be a bit slower than if the file were in one piece).

For the record, I have heard what I consider to be fairly reliable second-hand reports of defragmenting actually "fixing" a problem with some extremely heavily fragmented Exchange (or was it MS SQL?) server, due to a quirk in Exchange. Under no other circumstances has defragging actually fixed any real problems, other than slightly speeding up sequential file access, which is what it's designed to do.

[MatsP]

Stephen. I did say that defragging doesn't create more space... Just shuffles things around on the disk. And the ONLY benefit of this is that the files are a little bit faster to read/write, because the blocks aren't spread all over the place. This is particularly important on FAT-file-systems, because of the way that the file-allocation is done [1]. NTFS, OS/2's HPFS, Linux's ExtFS[2,3] etc have a better way to manage files, and are less prone to fragmenting files.

Checkdisk will make sure that all links between the directory structure (which holds the name, date, size, first block, etc of the file) and the actual map of used sectors (FAT - File Allocation Table, or the corresponding table(s) in another file-system type [NTFS for example uses a different system, and a Unix file-system has another system again.]). In general, this shouldn't happen, but if the system crashes, a file-segment may have been written to the disk and marked as "used", but the entry in the directory structure has not been correctly updated, which means that you have a "lost file". Again, NTFS and other advanced file-systems are better at keeping the file-system consistant than FAT. FAT was originally designed for floppy-disks with 360KB capacity, and then extended to cope with hard-disks of a few megabytes, and eventually extended again to cope with GB hard-disks. It's great for small media because it's possible to use a small table to keep track of the disk-content. But for big disks, there are better ways. NTFS isn't perfect, but it's certainly A LOT better than FAT. It also keeps the filenames sorted in the directory structure, so if you want to find the file called mmm.c in a 2000-file directory, there's no need to look through every single entry until it's found [averaging 1000 "looks"], but a "mid-point search" can be used, and the file can be found in no more than 11 "looks".

[1] FAT, because it's using a table to track which blocks on the disk are in use and which aren't, and this table can be quite large [as per Rune's posts regarding 8GB flash cards], it's time-(and/or memory-)consuming to search the whole table to find a suitable space for a big file. So the first available block is always used in FAT. This means that if you erase some old file that was created at an early time in the system [i.e. low block number], the block(s) of that file will be used before any other space. So if you constantly add and remove files from the system, you will eventually end up having a lot of small segments of files all over the disk. This is what disk-defragmenters fix - they put all the "bits of file" into larger chunks, so that as far as possible, the file-segments are all in one contiguous sequence of blocks.

--
Mats

[Rune]

NTFS, OS/2's HPFS, Linux's ExtFS[2,3] etc have a better way to manage files, and are less prone to fragmenting files.

AFAICT you need co-operation from the software you use. E.g. we have some Windows based servers running mySQL and it insists on growing its files byte-by-byte... Or that's what I think as it ends up with some serious fragmentation after a short while... (we typically add a couple of MB per day to the database, and there are processes in the background that also writes to the same volume as the dbms, thus... serious fragementation)

If you use xcopy to copy files, it will write out the files in one piece (if there is a contiguous block big enough). But I'm pretty sure it'll do that with FAT based partitions too. (this scenario is pretty simple though -- presumably the others handle more complex situations better)

I guess the filesystem driver could assume that a big file is likely to grow (and thus reserve space in advance), but... You could have lots of big files on a volume, so which parts should the filesystem prioritize?

Not that it matters much. But I am curious as to how other filesystems solve this. (assuming they do)

I think the neatest thing about a non-fragmented volume is that it is much easier to undelete files. (again; given FAT -- I do not know much about the inner workings of modern filesystems at this level, the only thing I know is that I like 'em better)

But as jliechty points out, you can't fix integrity problems by defragmenting the disk. However, most tools would force a chkdsk first (or similar). They pretty much have to... (in the case of FAT atleast)