{ Berbagi, Menulis, Dan Mengajar } Ilmu…

Linux files systems work great. Especially ext3 is a mature file system and very robust. With journaling turned the filesystems stays consistent also on an unexpected power loss or system crash (wich normally never happens on linux ) and you don’t need long disk checks to repair the file system in such a case. There is almost no disk fragmentation and a lot of more cool features … But the best linux feature is: if you remove a file or directory it’s gone It’s not moved to trash, its not marked as deleted, there are no annoying messageboxes, it just does what you say. I love that!
But from time to time also experienced users make mistakes and you delete something wrong. As Murphy’s law says: “if anything can go wrong it will” and so you will not have a copy of the deleted files and the backup will be outdated. So what to do now?
I want to explain the procedure by using the problem I had as example.

How the trouble came up for me

As usual I was working on my Gentoo Linux laptop. I just wanted to add a new folder with sources to my subversion repository. I wanted to add first the folder, then some files within the folder…

Adding a folder to svn is simple, sometimes too simple…
svn add uaconsoleclientsample
Now I saw that this command added all files recursively, not just the folder. That means a lot of temporary files where added too.
The good thing with svn is, everthing you change is just done on your local working copy and not on the server. So it’s not a problem until you commit your changes. So lets remove the wrong files again before committing the changes.

Removing the files…
svn remove uaconsoleclientsample
svn: Use –force to override this restriction
svn: ‘uaconsoleclientsample’ has local modifications
Ok, remove didn’t work, use –force.

svn –force remove uaconsoleclientsample
Bad idea!!! The “svn remove” command not only removes files from scheduled add operations like I assumed. It removes it also from your local working copy.

So now my uncommited files were lost. This was the time where ext3grep came into play.

Recover lost files with ext3grep

I knew every new file could overwrite the data of my deleted files. So quickly unmount the partition. (The great thing, linux users have multiple partitions. We can unmount /home and continue to work as root)

Unmounting /home
Close my X Session
Alt-F1 -> log in as root
/etc/init.d/xdm stop
umount /home Cannot umount, files still in access
lsof | grep home -> artsd is still accessing home
killall artsd
umount /home Succeeded.
OK, done. Nothing can delete any more data now. Now relax, breath through and go to another Desktop PC, start the browser and ask google for a solution. -> ext3grep Ok, lets try this out. Login into laptop from the desktop, that’s more comfortable, just copy paste URLs to the shell etc. ssh -l root laptop

Installing ext3grep…
wget http://ext3grep.googlecode.com/files/ext3grep-0.7.0.tar.gz
tar -xvzf ext3grep-0.7.0.tar.gz
cd ext3grep
./configure
make
cd src
./ext3grep –help
Done, ext3grep is built and working

Lets search for my deleted folder:My laptop has an LUKS encrypted hard disk, and I devided this encrypted partition using LVM. For that reason my home partition is not a physical one like /dev/hda, but /dev/mapper/vg-home. Replace that with your partition you are using.
lt_gergap src # ./ext3grep /dev/mapper/vg-home –search uaconsoleclientsample
Running ext3grep version 0.7.0
Number of groups: 240
Minimum / maximum journal block: 1547 / 35890
Loading journal descriptors… sorting… done
Journal transaction 1202732 wraps around, some data blocks might have been lost of this transaction.
Number of descriptors in journal: 30993; min / max sequence numbers: 1202700 / 1204860
Blocks containing “uaconsoleclientsample”: 240017 242352 242355 242356 (allocated) 242357
(allocated) 242358 (allocated) 242359 255079 (allocated) 336393 336518 336526 395434 395435
395457 (allocated) 737282 (allocated) 984250 1346129 1868670 (allocated) 1869273 (allocated)
1950436 3915933 3915935 4069411 4087953 4216611 4292193 4292196 4292275 4530219 4538370
4538371 4538372 4538376 4538378 4538382 4538385 4543743 4543750 4543752 4544514 4544517
4544528 4544539 4550683 4550707 4655509 4655533 4670417 4670423 4689385 4689746 4785120
5046823 6525842 (allocated) 7370457 7805912
Wow, there are a lot of blocks. Why are some allocated? Lets check them all.

lt_gergap src # ./ext3grep /dev/mapper/vg-home –ls –block 240017
Running ext3grep version 0.7.0
Number of groups: 240
Minimum / maximum journal block: 1547 / 35890
Loading journal descriptors… sorting… done
Journal transaction 1202732 wraps around, some data blocks might have been lost of this transaction.
Number of descriptors in journal: 30993; min / max sequence numbers: 1202700 / 1204860
Group: 7
Block 240017 is Unallocated.
Hmm, seems not to be a directory, at least it contains nothing. Try the next one …

This operation takes some time and there are a lot of blocks. I’m lazy and before I do something twice I write a little script to automate this procedure.

1 #!/bin/sh
2

7 done

This script does the same procedure for each block in the list, shows the output on console and writes it to the file output.txt using tee. This way you can recheck the result later also in vim.

Analyzing the file output.txt I have seen only one interesting block:

…
Running ext3grep version 0.7.0
Number of groups: 240
Minimum / maximum journal block: 1547 / 35890
Loading journal descriptors… sorting… done
Journal transaction 1202732 wraps around, some data blocks might have been lost of this transaction.
Number of descriptors in journal: 30993; min / max sequence numbers: 1202700 / 1204860
Group: 57

Block 1869273 is a directory. The block is Allocated

.– File type in dir_entry (r=regular file, d=directory, l=symlink)
| .– D: Deleted ; R: Reallocated
Indx Next | Inode | Deletion time Mode File name
==========+==========+—————-data-from-inode——+———–+=========
0 1 d 657336 drwxr-xr-x .
1 2 d 657111 drwxr-xr-x ..
2 3 d 657337 drwxr-xr-x .svn
3 4 d 657345 drwxr-xr-x qtestclient
4 5 d 657354 drwxr-xr-x uaserverc
5 6 d 657444 drwxr-xr-x performanceclient
6 7 d 657534 drwxr-xr-x uademoserver
7 8 d 657687 drwxr-xr-x comdaproxy
8 9 d 657696 drwxr-xr-x examples_bin
9 10 d 657903 drwxr-xr-x uaconsoleclient
10 11 d 672247 drwxr-xr-x uaconsoleclientcpp
11 12 d 672766 drwxr-xr-x demoserver
12 13 d 672784 drwxr-xr-x uaserver
13 14 d 657946 drwxr-xr-x ascserver
14 15 d 672981 drwxr-xr-x server_ads
15 16 d 657947 drwxr-xr-x uaexpert
16 18 d 657949 drwxr-xr-x uasimpleclient
17 18 d 675465 D 1215690762 Thu Jul 10 13:52:42 2008 drwxr-xr-x bin
18 end d 656495 drwxr-xr-x uaconsoleclientsample
…
Yes, there it is. Let’s check the contents of this folder.

lt_gergap src # ./ext3grep /dev/mapper/vg-home –ls –inode 656495
Running ext3grep version 0.7.0
Number of groups: 240
Minimum / maximum journal block: 1547 / 35890
Loading journal descriptors… sorting… done
Journal transaction 1202732 wraps around, some data blocks might have been lost of this transaction.
Number of descriptors in journal: 30993; min / max sequence numbers: 1202700 / 1204860
Inode is Allocated
Loading vg-home.ext3grep.stage2……………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………….
WARNING: Rejecting a dir_entry (block 5714387) because it contains legal but unlikely characters.
Use –ls –block 5714387 to examine this possible directory block.
If it looks like a directory to you, and ’24’
looks like a filename that might belong in that directory, then add
–accept=’24’ as commandline parameter AND remove both stage* files!
……………………………………………………………………………………………………………………………………………………………………. done
The first block of the directory is 1348508.
Inode 656495 is directory “kde4/work/opcua/applications/uaconsoleclientsample”.
Directory block 1348508:
.– File type in dir_entry (r=regular file, d=directory, l=symlink)
| .– D: Deleted ; R: Reallocated
Indx Next | Inode | Deletion time Mode File name
==========+==========+—————-data-from-inode——+———–+=========
0 1 d 656495 drwxr-xr-x .
1 11 d 657336 drwxr-xr-x ..
2 4 r 2328216 D 1215690762 Thu Jul 10 13:52:42 2008 rrw-r–r– Makefile
3 4 r 2263308 D 1215690762 Thu Jul 10 13:52:42 2008 rrw-r–r– main.cpp
4 5 r 2328213 D 1215690762 Thu Jul 10 13:52:42 2008 rrw-r–r– qdevelop-settings.db
5 11 r 2328221 D 1215690762 Thu Jul 10 13:52:42 2008 rrw-r–r– uaconsolesampleclient.pro
6 7 r 2328215 D 1215690762 Thu Jul 10 13:52:42 2008 rrw-r–r– main.o
8 9 d 675466 D 1215690762 Thu Jul 10 13:52:42 2008 drwxr-xr-x moc
9 11 r 2262157 D 1215690762 Thu Jul 10 13:52:42 2008 rrwxr-xr-x uaconsolesampleclient
10 11 d 675465 D 1215690762 Thu Jul 10 13:52:42 2008 drwxr-xr-x PKI
11 end d 656701 drwxr-xr-x .svn
12 end r 2263307 D 1215690762 Thu Jul 10 13:52:42 2008 rr–r–r– .main.cpp.swx
Yes, everything is there. Now just restore the files you want using the right inode.

lt_gergap src # ./ext3grep /dev/mapper/vg-home –restore-inode 2263308
Running ext3grep version 0.7.0
Number of groups: 240
Minimum / maximum journal block: 1547 / 35890
Loading journal descriptors… sorting… done
Journal transaction 1202732 wraps around, some data blocks might have been lost of this transaction.
Number of descriptors in journal: 30993; min / max sequence numbers: 1202700 / 1204860
Restoring inode.2263308
mv RESTORED_FILES/inode.2263308 /home/kde4/work/opcua/applications/uaconsoleclientsample/main.cpp
lt_gergap src # ./ext3grep /dev/mapper/vg-home –restore-inode 2328221
Running ext3grep version 0.7.0
Number of groups: 240
Minimum / maximum journal block: 1547 / 35890
Loading journal descriptors… sorting… done
Journal transaction 1202732 wraps around, some data blocks might have been lost of this transaction.
Number of descriptors in journal: 30993; min / max sequence numbers: 1202700 / 1204860
Restoring inode.2328221
mv RESTORED_FILES/inode.2328221 /home/kde4/work/opcua/applications/uaconsoleclientsample/uaconsolesampleclient.pro That’s all. ext3grep did a great job for me. Thanks to the developers of that tool.

Use svn the correct way
Adding just a folder to svn would have been easy. Just use the -N option to work non recursive way.

svn add -N uaconsoleclientsample