├── doc ├── img │ ├── sata-wb.jpg │ ├── dlt-lift.jpg │ ├── error-dd.png │ ├── omimgr-1.png │ ├── omimgr-2.png │ ├── dds-insert.jpg │ ├── dds-protect.jpg │ ├── dds3-jumpers.jpg │ ├── diskimgr-1.png │ ├── diskimgr-2.png │ ├── dlt-insert.jpg │ ├── dlt-inserted.jpg │ ├── dlt-loaded.jpg │ ├── dlt-protect.jpg │ ├── dlt-unloaded.jpg │ ├── flash-eject.png │ ├── omimgr-files.png │ ├── tapeimgr-1.png │ ├── tapeimgr-2.png │ ├── diskimgr-files.png │ ├── hdd-guymager1.png │ ├── hdd-guymager2.png │ ├── omimgr-errors.png │ ├── omimgr-errors2.png │ ├── omimgr-success.png │ ├── tapeimgr-files.png │ ├── diskimgr-success.png │ ├── display-lightdm.png │ ├── dlt-lower-handle.jpg │ ├── dlt-wp-indicator.jpg │ ├── flash-guymager1.png │ ├── flash-guymager2.png │ ├── flash-guymager3.png │ ├── floppy-35-eject.png │ ├── tapeimgr-success.png │ └── write-protection.png ├── readme.md ├── bitcurator-setup.md ├── disable-automount.md ├── usb-flash.md ├── rough-notes │ ├── hardwareChecklist.md │ ├── floppy-notes.md │ ├── tapes-formats-status.csv │ └── tape-notes.md ├── extract-dumpfile.md ├── harddisk.md ├── tape-dlt.md ├── cdrom-dvd.md ├── tape-dds.md ├── floppy-35.md └── df-resources.md ├── README.md ├── scripts ├── README.md ├── filedetect-tree.sh ├── extractcontainer.sh └── createcdimage.sh └── LICENSE /doc/img/sata-wb.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/sata-wb.jpg -------------------------------------------------------------------------------- /doc/img/dlt-lift.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/dlt-lift.jpg -------------------------------------------------------------------------------- /doc/img/error-dd.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/error-dd.png -------------------------------------------------------------------------------- /doc/img/omimgr-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/omimgr-1.png -------------------------------------------------------------------------------- /doc/img/omimgr-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/omimgr-2.png -------------------------------------------------------------------------------- /doc/img/dds-insert.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/dds-insert.jpg -------------------------------------------------------------------------------- /doc/img/dds-protect.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/dds-protect.jpg -------------------------------------------------------------------------------- /doc/img/dds3-jumpers.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/dds3-jumpers.jpg -------------------------------------------------------------------------------- /doc/img/diskimgr-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/diskimgr-1.png -------------------------------------------------------------------------------- /doc/img/diskimgr-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/diskimgr-2.png -------------------------------------------------------------------------------- /doc/img/dlt-insert.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/dlt-insert.jpg -------------------------------------------------------------------------------- /doc/img/dlt-inserted.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/dlt-inserted.jpg -------------------------------------------------------------------------------- /doc/img/dlt-loaded.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/dlt-loaded.jpg -------------------------------------------------------------------------------- /doc/img/dlt-protect.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/dlt-protect.jpg -------------------------------------------------------------------------------- /doc/img/dlt-unloaded.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/dlt-unloaded.jpg -------------------------------------------------------------------------------- /doc/img/flash-eject.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/flash-eject.png -------------------------------------------------------------------------------- /doc/img/omimgr-files.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/omimgr-files.png -------------------------------------------------------------------------------- /doc/img/tapeimgr-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/tapeimgr-1.png -------------------------------------------------------------------------------- /doc/img/tapeimgr-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/tapeimgr-2.png -------------------------------------------------------------------------------- /doc/img/diskimgr-files.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/diskimgr-files.png -------------------------------------------------------------------------------- /doc/img/hdd-guymager1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/hdd-guymager1.png -------------------------------------------------------------------------------- /doc/img/hdd-guymager2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/hdd-guymager2.png -------------------------------------------------------------------------------- /doc/img/omimgr-errors.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/omimgr-errors.png -------------------------------------------------------------------------------- /doc/img/omimgr-errors2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/omimgr-errors2.png -------------------------------------------------------------------------------- /doc/img/omimgr-success.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/omimgr-success.png -------------------------------------------------------------------------------- /doc/img/tapeimgr-files.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/tapeimgr-files.png -------------------------------------------------------------------------------- /doc/img/diskimgr-success.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/diskimgr-success.png -------------------------------------------------------------------------------- /doc/img/display-lightdm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/display-lightdm.png -------------------------------------------------------------------------------- /doc/img/dlt-lower-handle.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/dlt-lower-handle.jpg -------------------------------------------------------------------------------- /doc/img/dlt-wp-indicator.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/dlt-wp-indicator.jpg -------------------------------------------------------------------------------- /doc/img/flash-guymager1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/flash-guymager1.png -------------------------------------------------------------------------------- /doc/img/flash-guymager2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/flash-guymager2.png -------------------------------------------------------------------------------- /doc/img/flash-guymager3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/flash-guymager3.png -------------------------------------------------------------------------------- /doc/img/floppy-35-eject.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/floppy-35-eject.png -------------------------------------------------------------------------------- /doc/img/tapeimgr-success.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/tapeimgr-success.png -------------------------------------------------------------------------------- /doc/img/write-protection.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KBNLresearch/forensicImagingResources/master/doc/img/write-protection.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## KB Forensic Imaging Resources 2 | 3 | This repo contains various resources and documentation related to an effort at setting up an experimental small-scale forensic imaging facility. 4 | -------------------------------------------------------------------------------- /scripts/README.md: -------------------------------------------------------------------------------- 1 | # Scripts 2 | 3 | This directory contains various scripts for extracting data from a variety of carriers, and for the subsequent processing of these data. 4 | 5 | ## Note on old tape reading scripts 6 | 7 | The old scripts for reading tapes are superseded by the *tapeimgr* tool, which is available here: 8 | 9 | -------------------------------------------------------------------------------- /doc/readme.md: -------------------------------------------------------------------------------- 1 | # Documentation 2 | 3 | ## Workflow descriptions 4 | 5 | - [3.5" floppy disk](./floppy-35.md) 6 | - [hard disk](./harddisk.md) 7 | - [DDS-1 tape](./tape-dds.md) 8 | - [DLT-IV tape](./tape-dlt.md) 9 | - [USB Flash drive](./usb-flash.md) 10 | - [CD-ROM / DVD](./cdrom-dvd.md) 11 | - [Extract contents of Unix dump file](./extract-dumpfile.md) 12 | 13 | ## Resources 14 | 15 | - [Digital forensics and web archaeology resources](./df-resources.md) 16 | 17 | ## Miscellaneous 18 | 19 | - [BitCurator setup notes](./bitcurator-setup.md) 20 | - [Various rough,unedited notes](./rough-notes/) 21 | -------------------------------------------------------------------------------- /doc/bitcurator-setup.md: -------------------------------------------------------------------------------- 1 | # BitCurator setup notes 2 | 3 | Below setup notes describe how to set up BitCurator with MATE as a replacement for the default GNOME desktop (useful for those who prefer a more traditional menu-based desktop). Tested with BitCurator 2.0.6. 4 | 5 | 1. Install BitCurator from the latest ISO at 6 | 7 | 2. Install the MATE desktop using the below commands: 8 | 9 | sudo apt update 10 | sudo apt install ubuntu-mate-desktop 11 | 12 | 3. When the *package configuration* dialog pops up, select the *lightdm* display manager: 13 | 14 | ![](./img/display-lightdm.png) 15 | 16 | 4. Once the installation has finished, reboot the machine. 17 | 18 | 6. To make the BitCurator-specific context menus work, copy the *scripts* folder in `/home/bcadmin/.local/share/nautilus` to `/home/bcadmin/.config/caja`: 19 | 20 | cp -a /home/bcadmin/.local/share/nautilus/scripts /home/bcadmin/.config/caja/scripts 21 | 22 | 7. Optionally replace the default MATE wallpaper (which annoyingly has a similar colour as the folder icons on the desktop!) with one of the wallpapers in `/usr/share/backgrounds`. -------------------------------------------------------------------------------- /doc/disable-automount.md: -------------------------------------------------------------------------------- 1 | 2 | # Disable automatic mounting of removable media 3 | 4 | In order to minimise any risks of accidental write actions to floppy disks, it is strongly suggested to disable automatic mounting of removable media. The exact command depends on the Linux desktop you're using. For the [MATE](https://mate-desktop.org/) desktop use this: 5 | 6 | ```bash 7 | gsettings set org.mate.media-handling automount false 8 | ``` 9 | 10 | For a [GNOME](https://www.gnome.org/) desktop use this command: 11 | 12 | ```bash 13 | gsettings set org.gnome.desktop.media-handling automount false 14 | ``` 15 | 16 | And for the [Cinnamon](https://projects.linuxmint.com/cinnamon/) desktop: 17 | 18 | ```bash 19 | gsettings set org.cinnamon.desktop.media-handling automount-open false 20 | ``` 21 | 22 | You can use the below command to verify the automount setting (MATE): 23 | 24 | ```bash 25 | gsettings get org.mate.media-handling automount 26 | ``` 27 | 28 | Or, for GNOME: 29 | 30 | ```bash 31 | gsettings get org.gnome.desktop.media-handling automount 32 | ``` 33 | 34 | And finally for Cinnamon: 35 | 36 | ```bash 37 | gsettings get org.cinnamon.desktop.media-handling automount-open 38 | ``` 39 | 40 | If all goes well, this will result in: 41 | 42 | ``` 43 | false 44 | ``` 45 | 46 | Please be aware that disabling the automount feature does not provide tamper-proof write blocking! It only works at the level of the operating system's default file manager, and it won't keep a user from manually mounting a device. Also, the *gsettings* command only works at the user level. This means that for someone who logs in with a different user name, the default automount setting applies (which means automount will be enabled). 47 | -------------------------------------------------------------------------------- /scripts/filedetect-tree.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Identify format of each file in directory tree with file(1) 4 | # Check command line args 5 | if [ "$#" -ne 2 ] ; then 6 | echo "Usage: filedetect-tree.sh rootDirectory outputFile" >&2 7 | exit 1 8 | fi 9 | 10 | if ! [ -d "$1" ] ; then 11 | echo "rootDirectory must be a directory" >&2 12 | exit 1 13 | fi 14 | 15 | # Root directory 16 | rootDir="$1" 17 | 18 | # Output file 19 | outFile="$2" 20 | 21 | # Delete output file if it exists already 22 | if [ -f $outFile ] ; then 23 | rm $outFile 24 | fi 25 | 26 | echo "Processing directory tree ..." 27 | 28 | # Write header fields to output file 29 | echo fileName,mimeTypeFile,matchFile > $outFile 30 | 31 | # Record start time 32 | start=`date +%s` 33 | 34 | # This works for filenames that contain whitespace using code adapted from: 35 | # http://stackoverflow.com/questions/7039130/bash-iterate-over-list-of-files-with-spaces/7039579#7039579 36 | 37 | while IFS= read -d $'\0' -r file ; do 38 | # File basename (used as filename hint by Tika in -H option) 39 | # In production workflow bName could be read from metadata, in case actual file doesn't have original name/extension 40 | bName=$(basename "$file") 41 | #mimeTypeFile=$(file --mime-type "$file" | cut -d':' -f2 | cut -d' ' -f2) 42 | mimeTypeFile=$(file "$file") 43 | 44 | if [ "$mimeTypeFile" == "application/octet-stream" ] 45 | then 46 | matchFile=0 47 | else 48 | matchFile=1 49 | fi 50 | 51 | echo $file,$mimeTypeFile,$matchFile >> $outFile 52 | done < <(find $rootDir -type f -name '*.dd' -print0) 53 | 54 | # Record end time 55 | end=`date +%s` 56 | 57 | runtime=$((end-start)) 58 | 59 | echo "Running time for processing directory tree:" $runtime "seconds" 60 | -------------------------------------------------------------------------------- /scripts/extractcontainer.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Extract contents of dump and tar containers 4 | # for dump files this requires some user input, 5 | # and permissions are fixed on output afterwards 6 | # Script must be run with sudo privilege 7 | 8 | if [ "$#" -ne 2 ] ; then 9 | echo "Usage: extractcontainer.sh fileIn dirOut" >&2 10 | exit 1 11 | fi 12 | 13 | if ! [ -f "$1" ] ; then 14 | echo "fileIn must be a file" >&2 15 | exit 1 16 | fi 17 | 18 | if ! [ -d "$2" ] ; then 19 | echo "dirOut must be a directory" >&2 20 | exit 1 21 | fi 22 | 23 | # Input file 24 | fileIn="$1" 25 | 26 | # Establish file type 27 | fTypeString=$(file $fileIn) 28 | 29 | if [[ $fTypeString == *"new-fs dump file"* ]]; then 30 | fileType="dump" 31 | fi 32 | 33 | if [[ $fTypeString == *"tar archive"* ]]; then 34 | fileType="tar" 35 | fi 36 | 37 | echo $fileType 38 | 39 | # Output directory (absolute path) 40 | dirOut=$(realpath "$2") 41 | 42 | # User directory (absolute path) 43 | dirUser=$(pwd) 44 | 45 | # Go to output directory 46 | cd $dirOut 47 | 48 | if [[ $fileType == "dump" ]]; then 49 | echo "Extracting dump file ..." 50 | # Source: https://docs.oracle.com/cd/E19253-01/817-5093/bkuprestoretasks-72504/index.html 51 | restore -xvf $fileIn . 52 | extractStatus=$? 53 | # Go back to user directory 54 | cd $dirUser 55 | 56 | echo "Fixing permissions on extracted files and directories ..." 57 | find $dirOut -type f -exec chmod 644 {} \; 58 | find $dirOut -type d -exec chmod 755 {} \; 59 | 60 | fi 61 | 62 | if [[ $fileType == "tar" ]]; then 63 | echo "Extracting TAR archive ..." 64 | # Run tar 65 | tar -xf $fileIn 66 | extractStatus=$? 67 | # Go back to user directory 68 | cd $dirUser 69 | 70 | # Transfer ownership of dirOut to default user 71 | # (only affects top-level directory) 72 | echo "Set owner of top-level directory ..." 73 | chown johan $dirOut 74 | 75 | #echo "Fixing permissions on extracted files and directories ..." 76 | #find $dirOut -type f -exec chmod 644 {} \; 77 | #find $dirOut -type d -exec chmod 755 {} \; 78 | fi 79 | 80 | # Notify user 81 | echo "Done, exit status of extract command was "$extractStatus -------------------------------------------------------------------------------- /doc/usb-flash.md: -------------------------------------------------------------------------------- 1 | # USB flash drive 2 | 3 | ## Introduction 4 | 5 | [USB flash drive](https://en.wikipedia.org/wiki/USB_flash_drive). 6 | 7 | ## Hardware 8 | 9 | ### Write blocker 10 | 11 | |**Model**|[Tableau Forensic USB 3.0 Bridge](https://www.guidancesoftware.com/tableau/hardware/t8u)| 12 | |:--|:--| 13 | |**Connectors: Host (Left) Side**|USB 3.0 Standard-B connector| 14 | |**Connectors: Device (Right) Side**|USB 3.0 Standard-A connector| 15 | 16 | ## Connecting the write blocker 17 | 18 | 1. Hook up the write blocker to the workstation using the blue USB cable (use the left-hand port on the write blocker). 19 | 20 | 2. Connect the write blocker' s power supply to the DC In (at top) and make sure the power cord is plugged into to a power socket. 21 | 22 | ## Procedure for creating a disk image 23 | 24 | ### Guymager 25 | 26 | 1. Press the power button on the write blocker (if it is not switched on already). 27 | 28 | 2. Connect the flash drive to the right-hand USB connector on the write blocker. 29 | 30 | 3. Start Guymager. Result: 31 | 32 | ![](./img/flash-guymager1.png) 33 | 34 | 4. Right-click on the entry for the flash drive, and select *Acquire image*. 35 | 36 | 5. In the *Acquire image* dialog, make the following settings: 37 | 38 | - Set *File format* to *Linux dd raw*. 39 | - Uncheck the *Split image files* checkbox. 40 | - Select the destination directory for the disk image, and enter a file name (without extension). 41 | - Check *Calculate SHA-256* (and uncheck the *MD5* and *SHA-1* options). 42 | - Check *Verify image after acquisition*. 43 | 44 | ![](./img/flash-guymager2.png) 45 | 46 | 6. Press the *Start* button and watch the progress indicator. 47 | 48 | 7. If all went well, the *State* field will read *Finished - Verified & ok*. 49 | 50 | ![](./img/flash-guymager3.png) 51 | 52 | The above steps result in two files: 53 | 54 | - A file with extension *.dd*, which is the actual disk image. 55 | - A file with extension *.info*, which contains information about the imaging process. 56 | 57 | 8. Open a file manager window, right-click on the flash drive icon, and select *Eject*. 58 | 59 | ![](./img/flash-eject.png) 60 | 61 | 9. You can now safely remove the flash drive from the write blocker. 62 | -------------------------------------------------------------------------------- /doc/rough-notes/hardwareChecklist.md: -------------------------------------------------------------------------------- 1 | ## Hardware checklist 2 | 3 | |Hardware|In stock?|Comments| 4 | |:--|:--|:--| 5 | |Dedicated desktop for disk imaging / data extraction|1|Machine with BitCurator as main OS| 6 | |Separate desktop / laptop|?|Needed for looking up on-line documentation, and documenting the recovery process (lab book)| 7 | |Write blockers|-|SATA/IDE; USB; possibly SCSI (most vendors appear to have discontinued SCSI blockers)| 8 | |USB 3.5" floppy drives|1|[Doesn't work for e.g. Mac-formatted discs](https://porterolsen.wordpress.com/2016/06/15/accessing-mac-formatted-floppy-disks-without-the-kryoflux/); might not play well with USB write blockers| 9 | |5.25" floppy drives|1|I think we have at least 1 in storage, needs checking| 10 | |Controller for 5.25" floppy drive/ USB adapter|-|Several options are listed [here](https://www.archiveteam.org/index.php/Rescuing_Floppy_Disks). Many colleagues use [Kryoflux](https://www.kryoflux.com/); for 5.25" drives [Device Side Data's FC5025](http://www.deviceside.com/fc5025.html) looks like an interesting (and cheap) option| 11 | |ZIP drives|?|I think we have at least 1 in storage, needs checking| 12 | |Tape drive, DDS-1|1|[Digital Data Storage](https://en.wikipedia.org/wiki/Digital_Data_Storage) 13 | |Tape drive, DLT-IV|1|[Digital Linear Tape](https://en.wikipedia.org/wiki/Digital_Linear_Tape)| 14 | |Old, unimportant DDS-1 and DLT-IV tapes for testing|Yes, several|Needed for testing the tape readers, as we don't want to accidentally destroy any important tapes because the tape readers are not working properly!| 15 | |SCSI PCI card(s)|-|Needed for connecting SCSI tape drives. A bit tricky because of the plethora of SCSI variants and connectors; see also [here](http://qanda.digipres.org/1160/best-way-to-read-scsi-tape-drive-on-a-modern-pc)| 16 | |Various SCSI converters||See also the [SCSI Connector Guide](https://www.cablestogo.com/learning/connector-guides/scsi)| 17 | |Various cables and adapters||Useful resources are [The Cable Bible](https://amiaopensource.github.io/cable-bible/), [SCSI Connector Guide](https://www.cablestogo.com/learning/connector-guides/scsi) and [the SCSI articles on Paralan](http://www.paralan.com/aboutscsi.html)| 18 | |Power strips; spare power cables|-|| 19 | |Cans of compressed air for blowing away dust||| 20 | |Utility kit with screwdrivers||| 21 | |Camera (for documentation)|-|| 22 | |Grounded bench|-|Needed to for working with electronic components. Room may need other measures to reduce electrostatic discharge. See e.g. [this blog about Michigan University's Digital Preservation Lab](https://www.lib.umich.edu/blogs/bits-and-pieces/digital-preservation-lab-20)| 23 | |Anti-static wrist straps||| 24 | 25 | This checklist is partially based on [DIY Handboek voor Webarcheologie](https://hart.amsterdam/image/2017/11/17/20171116_freeze_diy_handboek.pdf) by Tjarda de Haan, Robert Jansma and Paul Vogel. -------------------------------------------------------------------------------- /doc/rough-notes/floppy-notes.md: -------------------------------------------------------------------------------- 1 | # Floppy notes 2 | 3 | Floppy drive: TEAC FD−55GFR−149U. This is one of the recommended drives to use with the FC5025 controller. Looks like jumper settings are OK out of the box. 4 | 5 | ## Compiling the drivers 6 | 7 | First install libusb using: 8 | 9 | sudo apt-get install libusb-dev 10 | 11 | 14 | 15 | Then compile the driver. Don't bother with compiling the GUI, as it needs an ancient version of GTK. 16 | 17 | ## Installation of drivers 18 | 19 | Copy the binaries (fcbrowse, fcformats, fcdrives and fcimage) to one of the *bin* directories. In my case I used the private *.local/bin directory in my home directory. Add this this directory to the PATH variable if necessary by adding below lines to the .profile file in the home directory: 20 | 21 | # set PATH so it includes user's private ~/.local/bin if it exists 22 | if [ -d "$HOME/.local/bin" ] ; then 23 | PATH="$HOME/.local/bin:$PATH" 24 | fi 25 | 26 | Then activate the new settings by running: 27 | 28 | source ~/.profile 29 | 30 | 63 | 64 | ## Using the tools 65 | 66 | Browsing the contents of an MS-DOS formatted 1200k floppy: 67 | 68 | fcbrowse -f msdos12 69 | 70 | BUT this gives the following result: 71 | 72 | fcbrowse: No devices found. 73 | 74 | Same for other format values. Under Windows the device is properly recognized, so there appears to be a problem with the compiled binaries. The most likely cause is that the docs state that the tools need version 0.1.12 of libusb, which is ancient (and AFAIK it cannot be installed on a modern Linux system without conflicts). 75 | 76 | **UPDATE**: 77 | 78 | See also: 79 | 80 | 81 | 82 | Detailed instructions on how to compile the software here; 83 | 84 | 85 | 86 | 87 | ## More fc5025 resources 88 | 89 | 90 | -------------------------------------------------------------------------------- /doc/rough-notes/tapes-formats-status.csv: -------------------------------------------------------------------------------- 1 | directory,fileName,format,extracted,encoding_errors,encerrors_retry,remarks 2 | tapes-DDS/1,file000001.dd,new-fs dump file (big endian),x,no,, 3 | tapes-DDS/1,file000002.dd,new-fs dump file (big endian),x,no,, 4 | tapes-DDS/1,file000003.dd,new-fs dump file (big endian),x,no,, 5 | tapes-DDS/2,file000001.dd,tar archive,x,no,, 6 | tapes-DDS/3,file000001.dd,new-fs dump file (big endian),x,no,, 7 | tapes-DDS/3,file000002.dd,new-fs dump file (big endian),x,no,, 8 | tapes-DDS/3,file000003.dd,new-fs dump file (big endian),x,no,, 9 | tapes-DDS/4,file000001.dd,tar archive,x,no,, 10 | tapes-DDS/5,file000001.dd,tar archive,x,no,, 11 | tapes-DDS/6,file000001.dd,new-fs dump file (big endian),x,no,, 12 | tapes-DDS/6,file000002.dd,new-fs dump file (big endian),x,no,, 13 | tapes-DDS/6,file000003.dd,new-fs dump file (big endian),x,yes,no, 14 | tapes-DDS/7,file000001.dd,data,n/a,no,, 15 | tapes-DDS/8,file000001.dd,POSIX tar archive (GNU),x,no,, 16 | tapes-DDS/9,file000001.dd,new-fs dump file (little endian),x,no,, 17 | tapes-DDS/9,file000002.dd,new-fs dump file (little endian),x,no,, 18 | tapes-DDS/9,file000003.dd,new-fs dump file (little endian),x,no,, 19 | tapes-DDS/9,file000004.dd,new-fs dump file (little endian),x,no,, 20 | tapes-DDS/10,file000001.dd,new-fs dump file (little endian),x,no,, 21 | tapes-DDS/10,file000002.dd,new-fs dump file (little endian),x,no,, 22 | tapes-DDS/10,file000003.dd,new-fs dump file (little endian),x,no,, 23 | tapes-DDS/10,file000004.dd,new-fs dump file (little endian),x,no,, 24 | tapes-DDS/11,file000001.dd,POSIX tar archive,x,no,, 25 | tapes-DDS/12,file000001.dd,new-fs dump file (big endian),x,no,, 26 | tapes-DDS/12,file000002.dd,new-fs dump file (big endian),x,no,, 27 | tapes-DDS/12,file000003.dd,new-fs dump file (big endian),x,no,, 28 | tapes-DDS/13,file000001.dd,new-fs dump file (big endian),x,no,, 29 | tapes-DDS/13,file000002.dd,new-fs dump file (big endian),x,no,, 30 | tapes-DDS/13,file000003.dd,new-fs dump file (big endian),x,yes,no, 31 | tapes-DDS/14,file000001.dd,tar archive,x,no,, 32 | tapes-DDS/15,file000001.dd,new-fs dump file (big endian),x,no,, 33 | tapes-DDS/15,file000002.dd,new-fs dump file (big endian),x,no,, 34 | tapes-DDS/15,file000003.dd,new-fs dump file (big endian),x,yes,no, 35 | tapes-DDS/16,file000001.dd,new-fs dump file (big endian),x,no,, 36 | tapes-DDS/16,file000002.dd,new-fs dump file (big endian),x,no,, 37 | tapes-DDS/16,file000003.dd,new-fs dump file (big endian),x,no,, 38 | tapes-DDS/17,file000001.dd,POSIX tar archive (GNU),x,no,, 39 | tapes-DDS/17,file000002.dd,POSIX tar archive (GNU),x,yes,no, 40 | tapes-DDS/18,file000001.dd,POSIX tar archive (GNU),x,no,, 41 | tapes-DDS/18,file000002.dd,POSIX tar archive (GNU),x,yes,no, 42 | tapes-DDS/19,file000001.dd,POSIX tar archive,x,no,, 43 | tapes-DLT/1,file000001.dd,new-fs dump file (big endian),x,yes,no, 44 | tapes-DLT/2,file000001.dd,new-fs dump file (big endian),x,yes,no, 45 | tapes-DLT/3,file000001.dd,empty,,,, 46 | tapes-DLT/4,file000001.dd,new-fs dump file (big endian),,,,"Multi-volume(?), inode not found on tape" 47 | tapes-DLT/5,file000001.dd,empty,,,, 48 | tapes-DLT/6,file000001.dd,new-fs dump file (big endian),,,, 49 | tapes-DLT/7,file000001.dd,new-fs dump file (big endian),,,, 50 | tapes-DLT/8,file000001.dd,empty,,,, 51 | tapes-DLT/9,file000001.dd,empty,,,, 52 | tapes-DLT/10,file000001.dd,empty,,,, 53 | tapes-DLT/11,file000001.dd,empty,,,, 54 | tapes-DLT/12,file000001.dd,new-fs dump file (big endian),,,, 55 | tapes-DLT/13,file000001.dd,empty,,,, 56 | tapes-DLT/14,file000001.dd,new-fs dump file (big endian),,,, 57 | -------------------------------------------------------------------------------- /doc/extract-dumpfile.md: -------------------------------------------------------------------------------- 1 | # Extract contents of Unix dump file 2 | 3 | ## Introduction 4 | 5 | [Unix dump](http://fileformats.archiveteam.org/wiki/Unix_dump) was a backup utility that was widely used in the '90s. This document describes how to extract the contents of a *dump* file to an empty directory. 6 | 7 | ## Software 8 | 9 | [**Restore**](https://linux.die.net/man/8/restore), which is part of *dump* package. It can be installed using: 10 | 11 | sudo apt install dump 12 | 13 | ## Hardware 14 | 15 | It is strongly recommended to extract *dump* files to an [*Ext4*](\https://en.wikipedia.org/wiki/Ext4)-formatted disk. Extracting to an [*NTFS*](https://en.wikipedia.org/wiki/NTFS) file system (which is the default used by most commercially sold USB disks) is likely to result in errors, as the names of directories and files inside a *dump* file may contain characters that are not compatible with *NTFS*, and as a result these files are not extracted! 16 | 17 | ## Procedure for extracting a dump file 18 | 19 |
20 | 21 | **CAUTION**: by default *restore* extracts the contents of a *dump* file to the system's root directory, i.e. it tries to recover a full backup. In nearly all modern-day cases this behaviour is unwanted, and it could even wreak havoc on your machine's file system. Extraction to a user-defined directory is possible, but it does require some user interaction (which limits the possibilities for automating the recovery procedure). 22 | 23 |
24 | 25 | 1. Create an empty directory, and then go to that directory in the command terminal, e.g.: 26 | 27 | mkdir file000002 28 | 29 | cd file000002 30 | 31 | 2. Run `restore` as sudo with the following command-line arguments[^1]: 32 | 33 | sudo restore -xvf ../../tapes-DDS/1/file000002.dd . 34 | 35 | Here, `../../tapes-DDS/1/file000002.dd` points to the dump file. Note that the final `.` argument defines the files *inside* the dump file that are to be extracted (here: all of them), it does *not* define an output path! 36 | 37 | 3. After a while the following prompt appears: 38 | 39 | You have not read any volumes yet. 40 | Unless you know which volume your file(s) are on you should start 41 | with the last volume and work towards the first. 42 | Specify next volume # (none if no more volumes): 43 | 44 | Now enter `1`. This will start the extraction process. 45 | 46 | 4. After a while another prompt appears: 47 | 48 | set owner/mode for '.'? [yn] 49 | 50 | Now enter `n` [^2] and wait for `restore` to finish. 51 | 52 | 5. Move one directory level up: 53 | 54 | cd .. 55 | 56 | 6. Finally set the permissions on the extracted files and directories using the following two commands (replacing `file000002` with the name of your extraction directory): 57 | 58 | sudo find ./file000002 -type f -exec chmod 644 {} \; 59 | 60 | sudo find ./file000002 -type d -exec chmod 755 {} \; 61 | 62 | Done! 63 | 64 | ## Troubleshooting 65 | 66 | In some cases you may get errors like this during extraction: 67 | 68 | restore: ./Answerbook/SS2INSTALL/Conformit�_aux__Normes_de_S�curit�: cannot create file: Invalid or incomplete multibyte or wide character 69 | 70 | This typically happens when extracting to *NTFS*-formatted disks. Extracting to an *Ext4* formatted disk should get rid of this problem. 71 | 72 | ## Resources 73 | 74 | - How to Restore Specific UFS Files Noninteractively (Oracle): 75 | 76 | - How to Restore UFS Files Interactively (Oracle): 77 | 78 | 79 | [^1]: If you don't run `restore` as sudo, extraction results in a flood of `chown: Operation not permitted` messages (the files *are* extracted though). 80 | 81 | [^2]: Needs further investigation, implications of this setting and what it does are not 100% clear to me. -------------------------------------------------------------------------------- /scripts/createcdimage.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | show_help () 4 | { # Show help message 5 | cat << EOF 6 | Usage: ${0##*/} [-h] [-d device] [-c command] [-n baseName] 7 | [-s suffix] [-t tries] [-e description] [-o notes] 8 | 9 | Create ISO image from CD or DVD. 10 | 11 | positional arguments: 12 | 13 | dirOut output directory 14 | 15 | optional arguments: 16 | 17 | -h display this help message and exit 18 | -d device optical drive (default: /dev/sr0) 19 | -c command command used to read the disc. Allowed values: 20 | readom / ddrescue (default: readom) 21 | -n baseName base name for image file (default: image) 22 | -s suffix suffix for image file (default: iso) 23 | -t tries number of tries (default: 4) 24 | -e description descriptive string or title (default: empty string) 25 | -o notes notes (default: empty string) 26 | EOF 27 | } 28 | 29 | # Initialize command line variables 30 | 31 | # Optical drive 32 | deviceName="/dev/sr0" 33 | command="readom" 34 | # Base name 35 | baseName="image" 36 | description="" 37 | notes="" 38 | sectorSize="2048" 39 | suffix="iso" 40 | tries="4" 41 | 42 | OPTIND="1" 43 | # Read variable from command line 44 | while getopts ":h:d:c:n:e:o:s:t:" opt; do 45 | case "$opt" in 46 | h) 47 | show_help 48 | exit 0 49 | ;; 50 | d) deviceName="$OPTARG" 51 | ;; 52 | c) command="$OPTARG" 53 | ;; 54 | n) baseName="$OPTARG" 55 | ;; 56 | e) description="$OPTARG" 57 | ;; 58 | o) notes="$OPTARG" 59 | ;; 60 | s) suffix="$OPTARG" 61 | ;; 62 | t) tries="$OPTARG" 63 | ;; 64 | *) 65 | show_help >&2 66 | exit 1 67 | ;; 68 | esac 69 | done 70 | shift "$((OPTIND-1))" 71 | 72 | # Check command line args 73 | if [ "$#" -ne 1 ] ; then 74 | show_help 75 | exit 1 76 | fi 77 | 78 | # Positional arguments 79 | # dirOut, normalise to absolute path 80 | dirOut="$(readlink -f $1)" 81 | 82 | if ! [ -d "$dirOut" ] ; then 83 | echo "ERROR: dirOut must be a directory" >&2 84 | exit 1 85 | fi 86 | 87 | # Unmount disk (probably not really needed, but just making sure) 88 | umount $deviceName 89 | 90 | # Construct command line 91 | if [ $command = "readom" ] ; then 92 | cdReadCommand="readom retries=$tries dev=$deviceName f=$dirOut/$baseName.$suffix" > $dirOut/$baseName."stdout" 2> $dirOut/$baseName."stderr" 93 | fi 94 | 95 | if [ $command = "ddrescue" ] ; then 96 | cdReadCommand="ddrescue -d -n -b $sectorSize -r$tries -v $deviceName $dirOut/$baseName.$suffix $dirOut/$baseName.log" > $dirOut/$baseName."stdout" 2> $dirOut/$baseName."stderr" 97 | fi 98 | 99 | # Start datetime 100 | dtStart=$(date --iso-8601=seconds) 101 | 102 | # Run command line 103 | $cdReadCommand 104 | 105 | # Exit code 106 | readExitCode="$?" 107 | 108 | if [ $readExitCode = 0 ] ; then 109 | exitOK=true 110 | else 111 | exitOK=false 112 | echo "Error: read process exited with error!" >&2 113 | fi 114 | 115 | # Compute checksum on image, store to file 116 | checksum=$(sha512sum $dirOut/$baseName.$suffix) 117 | echo $checksum > $dirOut/$baseName.$suffix."sha512" 118 | 119 | # End datetime 120 | dtEnd=$(date --iso-8601=seconds) 121 | 122 | # Write log file (JSON format) 123 | 124 | logFile=$dirOut/"metadata.json" 125 | 126 | echo "{" > $logFile 127 | echo \""acquisitionStart"\": \"$dtStart\", >> $logFile 128 | echo \""acquisitionEnd"\": \"$dtEnd\", >> $logFile 129 | echo \""fileName"\": \"$baseName.$suffix\", >> $logFile 130 | echo \""readCommand"\": \"$cdReadCommand\", >> $logFile 131 | echo \""readExitCode"\": $readExitCode, >> $logFile 132 | echo \""description"\": \"$description\", >> $logFile 133 | echo \""notes"\": \"$notes\", >> $logFile 134 | echo \""checksumType"\": \""SHA-512"\", >> $logFile 135 | echo \""checksum"\": \"$(echo $checksum | cut -d ' ' -f 1)\" >> $logFile 136 | echo "}" >> $logFile 137 | 138 | # Fix access permissions 139 | 140 | #chmod 777 $baseName.$suffix 141 | #chmod 777 $baseName.json 142 | #chmod 777 $baseName.$suffix."md5" 143 | #chmod 777 $baseName."stdout" 144 | #chmod 777 $baseName."stderr" 145 | -------------------------------------------------------------------------------- /doc/harddisk.md: -------------------------------------------------------------------------------- 1 | # Hard disk drive (internal) 2 | 3 | ## Introduction 4 | 5 | [Hard disk drive](https://en.wikipedia.org/wiki/Hard_disk_drive). 6 | 7 | ## Hardware 8 | 9 | ### Write blocker 10 | 11 | |**Model**|[Tableau Forensic SATA/IDE Bridge](https://www.guidancesoftware.com/tableau/hardware/t35u)| 12 | |:--|:--| 13 | |**Connectors: Host (Left) Side**|USB 3.0 Standard-B connector| 14 | |**Connectors: Device (Right) Side**|SATA signal connector, IDE signal connector, 4-pin 3M drive power connector (power to SATA or IDE hard disk)| 15 | |**SATA Device Compatibility**|SATA 1 or SATA 2 hard disk devices| 16 | |**IDE Device Compatibility**|Parallel ATA hard disk devices with Logical Block Addressing (LBA) support| 17 | 18 | ## Connecting the write blocker 19 | 20 | 1. Hook up the write blocker to the workstation using the blue USB cable (use the left-hand port on the write blocker). 21 | 22 | 2. Connect the write blocker' s power supply to the DC In (top left) and make sure the power cord is plugged into to a power socket. 23 | 24 | ## Procedure for creating a disk image 25 | 26 | ### Guymager 27 | 28 | 1. Connect the hard disk drive to one of the connectors at the right-hand side of the write blocker using the appropriate (SATA/IDE) cables. Note that for a SATA drive you need to connect two cables: one data cable and a power cable! See the image below: 29 | 30 | ![](./img/sata-wb.jpg) 31 | 32 | 2. Press the power button on the write blocker. 33 | 34 | 3. Start Guymager. Result: 35 | 36 | ![](./img/hdd-guymager1.png) 37 | 38 | 4. Right-click on the entry for the flash drive, and select *Acquire image*. 39 | 40 | 5. In the *Acquire image* dialog, make the following settings: 41 | 42 | - Set *File format* to *Linux dd raw*. 43 | - Uncheck the *Split image files* checkbox. 44 | - Select the destination directory for the disk image, and enter a file name (without extension). 45 | - Check *Calculate SHA-256* (and uncheck the *MD5* and *SHA-1* options). 46 | - Check *Verify image after acquisition*. 47 | 48 | ![](./img/hdd-guymager2.png) 49 | 50 | 6. Press the *Start* button and watch the progress indicator. 51 | 52 | 7. If all went well, the *State* field will read *Finished - Verified & ok*. 53 | 54 | ![](./img/hdd-guymager3.png) 55 | 56 | The above steps result in two files: 57 | 58 | - A file with extension *.dd*, which is the actual disk image. 59 | - A file with extension *.info*, which contains information about the imaging process. 60 | 61 | 62 | 63 | 8. Open a file manager window, right-click on the hdd drive icon, and select *Unmount*. 64 | 65 | ![](./img/hdd-unmount.png) 66 | 67 | 9. You can now safely remove the hard disk drive from the write blocker. 68 | 69 | 70 | 83 | 84 | ## Get information about the disk image 85 | 86 | From the terminal, type: 87 | 88 | disktype testsata.dd 89 | 90 | Result: 91 | 92 | --- testsata.dd 93 | Regular file, size 298.1 GiB (320072933376 bytes) 94 | DOS/MBR partition map 95 | Partition 1: 350 MiB (367001600 bytes, 716800 sectors from 2048, bootable) 96 | Type 0x07 (HPFS/NTFS) 97 | NTFS file system 98 | Volume size 350.0 MiB (367001088 bytes, 716799 sectors) 99 | Partition 2: 297.7 GiB (319703482368 bytes, 624420864 sectors from 718848) 100 | Type 0x07 (HPFS/NTFS) 101 | NTFS file system 102 | Volume size 297.7 GiB (319703481856 bytes, 624420863 sectors) 103 | 104 | So in this case our disk image contains 2 partitions: 105 | 106 | - A 350 MiB boot partition 107 | - A 298 GiB partition 108 | 109 | The second partition starts at sector 718848 (byte offset 512*718848 = 368050176). 110 | 111 | ## Export files from disk image 112 | 113 | To export the second partition: 114 | 115 | tsk_recover -a -o 718848 ./image/testsata.dd ./fsOut -------------------------------------------------------------------------------- /doc/tape-dlt.md: -------------------------------------------------------------------------------- 1 | # DLT-IV tape 2 | 3 | ## Introduction 4 | 5 | [Digital Linear Tape (DLT)](https://www.wikidata.org/wiki/Q1224765) is is a family of tape formats that were developed by Digital Equipment Corporation (DEC) from 1984 onwards. 6 | 7 | ## Hardware 8 | 9 | ### Tape reader 10 | 11 | |**Model**|[IBM 7205 440](https://web.archive.org/web/20181101132449/https://www-01.ibm.com/common/ssi/cgi-bin/ssialias?infotype=DD&subtype=SM&htmlfid=649/ENUS7205-440)| 12 | |:--|:--| 13 | |**Media**|DLT IV (and possibly DLT III)| 14 | |**SCSI Signaling Type**|Low Voltage Differential (LVD)| 15 | |**Interface**|Fast Wide SCSI| 16 | |**Connector**|68 pin HD D-Sub (HD-68) (see also 5th from top [here](https://web.archive.org/web/20180606154144/http://www.paralan.com/sediff.html))| 17 | |**Cable**|At the outgoing side is a male VHDCI SCSI connector (bottommost [here](https://web.archive.org/web/20180606154144/http://www.paralan.com/sediff.html))| 18 | |**Workstation connection**|Can be connected directly with workstation's SCSI controller (which has a female VHDCI connector)| 19 | |**Documentation**|
  • [IBM 7205 Model 440 40 GB External Digital Linear Tape Drive Enhances Data Storage](https://web.archive.org/web/20181101131555/https://www-01.ibm.com/common/ssi/rep_ca/2/897/ENUS101-062/ENUS101-062.PDF)
  • [7205 Model 440 Digital Linear Tape Drive Setup and Operator Guide](https://web.archive.org/web/20181101132208/http://ps-2.kev009.com/basil.holloway/ALL%20PDF/a4100501.pdf)
| 20 | 21 | ### SCSI controller 22 | 23 | |**Model**|[Adaptec SCSI Card 29320LPE](https://web.archive.org/web/20161203082125/https://storage.microsemi.com/en-us/support/scsi/u320/asc-29320lpe/)| 24 | |:--|:--| 25 | |**Bus System Interface Type**|[PCI Express](https://www.wikidata.org/wiki/Q206924) x1| 26 | |**External Connectors**|[68-pin VHDCI](https://www.wikidata.org/wiki/Q4052507)| 27 | |**Data Transfer Rate**|Up to 320 MByte/sec.| 28 | 29 | #### Note on bracket height 30 | 31 | Note that by default the controller has a standard height (120 mm) bracket that won't fit into a low-profile (79.2 mm) slot! When these controllers were sold new, they came with a replacement low-profile bracket, but these are often not included with used ones. The low-profile replacement brackets are sometimes sold separately on eBay. 32 | 33 | ## Connecting / disconnecting the tape reader 34 | 35 | Always make sure that both the tape reader and the computer are switched off before connecting or disconnecting. Not doing so may cause damage to the computer, the tape reader or both. 36 | 37 | ## Cleaning cartridges 38 | 39 | The drive needs to be cleaned periodically with a cleaning cartridge. 40 | 41 | ## Software 42 | 43 | [tapeimgr](https://github.com/KBNLresearch/tapeimgr), running in [BitCurator](https://bitcurator.net/). 44 | 45 | ## Procedure for reading a tape 46 | 47 | 1. Check the write-protect tab on the bottom of the cartridge, and slide it to the leftmost position. The orange indicator must be visible: 48 | 49 | ![](./img/dlt-protect.jpg) 50 | 51 | 2. Lift the cartridge insert/release handle: 52 | 53 | ![](./img/dlt-lift.jpg) 54 | 55 | 3. Insert the tape cartridge into the drive. Make sure the printed side is on top, and that the tape is inserted in the direction of the arrow symbol: 56 | 57 | ![](./img/dlt-insert.jpg) 58 | 59 | 4. Lower the cartridge insert/release handle: 60 | 61 | ![](./img/dlt-lower-handle.jpg) 62 | 63 | The green "tape in use" indicator light (2nd from the top) now starts blinking. Once the tape is completely loaded, the blinking stops and the indicator light is lit solid. 64 | 65 | 5. Check that the green "write-protected" indicator light (top-right) is lit: 66 | 67 | ![](./img/dlt-wp-indicator.jpg) 68 | 69 | 6. Create an empty directory for storing the extracted tape contents. 70 | 71 | 7. Start *tapeimgr* from the OS's main menu (in Ubuntu 18.04 the *tapeimgr* item is located under *System Tools*), or by clicking the *tapeimgr* shortcut on the desktop. 72 | 73 | ![](./img/tapeimgr-1.png) 74 | 75 | 8. Use the *Select Output Directory* button to navigate to the empty directory created under step 6. Click on the *UUID* button to generate a unique identifier. Use the *Description* field to enter a description of the tape (e.g. the title that is written on its label or inlay card). Optionally, use the *Notes* field to record anything else worth mentioning about the tape (e.g. if the *Description* is ambiguous because the writing on the tape's label is illegible, make a note of it here). 76 | 77 | 9. Press the *Start* button to start the extraction, and then wait for *tapeimgr* to finish (depending on the amount of data on the tape this may take several hours). You can monitor the progress of the extraction procedure in the progress window: 78 | 79 | ![](./img/tapeimgr-2.png) 80 | 81 | Note that the screen output is also written to a log file in the output directory. A prompt appears when the extraction has finished: 82 | 83 | ![](./img/tapeimgr-success.png) 84 | 85 | If the extraction finished without any errors, the output directory now contains the following files: 86 | 87 | ![](./img/tapeimgr-files.png) 88 | 89 | Here, **file000001.dd** through **file000003.dd** are the extracted files; **checksums.sha512** contains the SHA512 checksums of the extracted files, **metadata.json** contains some basic metadata and **tapeimgr.log** is the log file. 90 | 91 | 10. When *tapeimgr* has finished, wait until the "tape in use" indicator light stops blinking, and the green "operate handle" light (bottom-right) comes on. Now lift the cartridge insert/release handle: 92 | 93 | ![](./img/dlt-unloaded.jpg) 94 | 95 | 11. Take out the tape cartridge. All done! 96 | 97 | Make sure to lower the insert/release handle again at the end of your tape reading session (i.e. before switching off the tape reader). 98 | -------------------------------------------------------------------------------- /doc/cdrom-dvd.md: -------------------------------------------------------------------------------- 1 | # CD-ROM / DVD 2 | 3 | ## Introduction 4 | 5 | [CD-ROMs](https://www.wikidata.org/wiki/Q7982) (including [CD-Rs](https://www.wikidata.org/wiki/Q420778)) and [DVDs](https://www.wikidata.org/wiki/Q5294) (including [DVD-Rs](https://www.wikidata.org/wiki/Q1071697)) are optical media carriers that contain digital data. The workflow described here is only suitable for single-session CD-ROMs and DVDs. It cannot be used for audio CDS, or multisession discs (the tools used only recognise the first session on a disc, and will igniore any remaining sessions). 6 | 7 | ## Hardware 8 | 9 | ### CD / DVD reader 10 | 11 | Can be an internal or external drive, or both. Having multiple drives can be useful for recovering data from defective discs. It is not uncommon to find that sectors give read errors in one drive are read correctly by another one (and vice versa), and the software workflow can take advantage of this (see below). 12 | 13 | ## Software 14 | 15 | [omimgr](https://github.com/KBNLresearch/omimgr), running in [BitCurator](https://bitcurator.net/). 16 | 17 | ## Procedure for reading a disc 18 | 19 | 1. Start *omimgr* from the OS's main menu (in Ubuntu 18.04 the *omimgr* item is located under *System Tools*). 20 | 21 | ![](./img/omimgr-1.png) 22 | 23 | 2. Use the *Select Output Directory* button to navigate to an empty directory where the disc image and its associated metadata will be stored (or create a new directory from the dialog). 24 | 25 | 3. If you are using an external CD or DVD drive, you may need to change the *Optical device* value accordingly (a typical value for an external device would be `/dev/sr1`). 26 | 27 | 4. Leave *Read method*, *Retries*, *Direct disc mode* and *Auto-retry* at their default values. 28 | 29 | 5. Click on the *UUID* button to generate a unique identifier. 30 | 31 | 6. Change the *Prefix* value to the name you want to give to the image file, and use the *Description* field to enter a description of the disc (e.g. the title that is written on its label or inlay card). Optionally, use the *Notes* field to record anything else worth mentioning about the disc (e.g. if the *Description* is ambiguous because the writing on the disc is illegible, make a note of it here). 32 | 33 | 7. Insert the disc into the drive, close it, and wait for the disc to mount (a file manager window pops up when the disc is mounted ). 34 | 35 | 9. Press the *Start* button to start imaging, and then wait for *omimgr* to finish. You can monitor the progress of the imaging procedure in the progress window: 36 | 37 | ![](./img/omimgr-2.png) 38 | 39 | Note that the screen output is also written to a log file in the output directory. If the imaging finished without any errors, the following prompt appears: 40 | 41 | ![](./img/omimgr-success.png) 42 | 43 | The output directory now contains the following files: 44 | 45 | ![](./img/omimgr-files.png) 46 | 47 | Here, **disc.iso** is the ISO image; **checksums.sha512** contains the SHA512 checksum of the image file, **metadata.json** contains some basic metadata and **omimgr.log** is the log file. 48 | 49 | 10. If the imaging procedure *did* result in any errors, follow the *Additional steps in case of errors* section below. Otherwise, take out the disc. 50 | 51 | 52 | ## Additional steps in case of errors 53 | 54 | If the reading of the disc with *readom* resulted in errors, the following prompt will appear: 55 | 56 | ![](./img/omimgr-errors.png) 57 | 58 | If this happens, follow the steps below. 59 | 60 | 1. Have a careful look at the text you see in the progress window. In particular, look at the error message that is reported by *readom*, and the value of *imageTruncated*. 61 | 62 | ![](./img/omimgr-errors2.png) 63 | 64 | In this case we see that only 1 sector could not be read (`The following 1 sector(s) could not be read correctly`). This is a common problem with recordable CDs, which often result in problems with the final 1 or 2 sectors. In most cases these sectors do not contain any meaningful data. This is confirmed by the value of *imageTruncated* (`imageTruncated: False`), which means the imaging result is most likely OK, in which case it's safe to click on *No*, and take out the disc. 65 | 66 | 2. If *readom* reports more extensive problems, and the value of *imageTruncated* is *True*, click *Yes*. *Omimgr* will now delete the current image file, and retry imaging the disc with the *ddrescue* tool (which is often better at recovering data from defective media). Note that, depending on the condition of the disc, *ddrescue* may need up to several hours to read it! 67 | 68 | 3. If *omimgr* still reports errors after the first pass with *ddrescue*, you can run additional passes to improve the result. Try re-running it in *Direct disc* mode (which can be selected from *omimgr*'s interface). Another useful technique is to run additional *ddrescue* passes with different optical devices (e.g. one or more external USB drives). 69 | 70 | 71 | 72 | 4. Take out the disc. All done! 73 | 74 | ## Interrupting readom or ddrescue 75 | 76 | Running imaging processes can be stopped by pressing the *Interrupt* button. This is particularly useful for *ddrescue* runs, which may require many hours for discs that are badly damaged. Note that interrupting *ddrescue* will not result in any data loss, and interrupted runs can be resumed at a later time (see below). Interrupting *readom* will generally result in an unreadable ISO image, and is not advised. 77 | 78 | ## Resuming an interrupted ddrescue run 79 | 80 | Follow these steps to resume a *ddrescue* run that was previously interrupted: 81 | 82 | 1. After launching *omimgr*, set the output directory to the directory of the interrupted run. 83 | 84 | 2. Set **Read method** to *ddrescue*. 85 | 86 | 3. Click on the **Load existing metadata** button; this loads the previously entered *Prefix*, *Extension*, *Identifier*, *Description* and *Notes* values. 87 | 88 | 4. Hit the **Start** button. Now *ddrescue* will simply pick up on where the interrupted run stopped. 89 | -------------------------------------------------------------------------------- /doc/tape-dds.md: -------------------------------------------------------------------------------- 1 | # DDS tape 2 | 3 | ## Introduction 4 | 5 | [Digital Data Storage (DDS)](https://www.wikidata.org/wiki/Q1224723) is a family of tape formats that are based on [digital audio tape](https://www.wikidata.org/wiki/Q597615) (DAT). 6 | 7 | ## Hardware 8 | 9 | ### Tape readers 10 | 11 | |**Model**|[HP SureStore DAT8](https://web.archive.org/web/20180606153928/http://www.hp.com/ecomcat/hpcatalog/specs/S4112B.htm)| 12 | |:--|:--| 13 | |**Media**|DDS-2 (8 GB), DDS-1 (2.6 GB and 4 GB)| 14 | |**Interface**|Single-ended, narrow SCSI-2| 15 | |**Connector**|50-contact, centronics-type connector ("SCSI-1 Connector" AKA "Alternative 2, A-cable connector"), female (see also the topmost connector [here](https://web.archive.org/web/20180606154144/http://www.paralan.com/sediff.html))[^1]| 16 | |**Cable**|Attached to the device is a [Single Ended HD68 TO LD50 SCSI Cable](https://web.archive.org/web/20180606100950/http://www.itinstock.com/hp-c5665-61001-single-ended-hd68-to-ld50-scsi-cable-1-meter-40938-p.asp). The connector other end is a 68-pin DB68 (MD68) male connector (also known as High-Density or HD 68 and Half-Pitch or HP68)| 17 | |**Workstation connection**|With this [HD68 to VHDCI adapter](https://web.archive.org/web/20181002103944/https://www.ramelectronics.net/sm-044-r.aspx) the reader can be hooked up to the workstation's SCSI controller (which has a female VHDCI connector)| 18 | |**Documentation**|
  • [HP Surestore and StorageWorks DAT - HP DAT Accessories and Part Numbers](https://web.archive.org/web/20181101135618/https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-lpg50205)
  • [HP SureStore DAT - Sun Workstation Configuration](https://web.archive.org/web/20181101140041/https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-lpg51069)
  • [HP StorageWorks DAT - Troubleshooting for HP StorageWorks DAT8, DAT24, DAT40, DAT72, and DAT160 SCSI and USB Devices](https://web.archive.org/web/20190128155319/https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c00057819)
| 19 | 20 | |**Model**|[HP StorageWorks DAT 72 (Q1523A)](https://web.archive.org/web/20190311120448/https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c00624867)| 21 | |:--|:--| 22 | |**Media**|DAT 72, DDS-4, DDS-3| 23 | |**Interface**|Wide Ultra SCSI-3 (LVD/SE)| 24 | |**Connector**|68-pin wide LVDS, thumbscrew| 25 | |**Cable**|| 26 | |**Workstation connection**|| 27 | |**Documentation**|
  • [HP StorageWorks DAT tape drive, getting started guide, DAT 40e, DAT 72e](https://web.archive.org/web/20190311115921/http://h10032.www1.hp.com/ctg/Manual/c00070850.pdf)
  • [HP StorageWorks DAT - Troubleshooting for HP StorageWorks DAT8, DAT24, DAT40, DAT72, and DAT160 SCSI and USB Devices](https://web.archive.org/web/20190128155319/https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c00057819)
| 28 | 29 | [^1]: This is somewhat strange, since the specs explicitly state the drive has a SCSI-2 interface 30 | 31 | ### SCSI controller 32 | 33 | |**Model**|[Adaptec SCSI Card 29320LPE](https://web.archive.org/web/20161203082125/https://storage.microsemi.com/en-us/support/scsi/u320/asc-29320lpe/)| 34 | |:--|:--| 35 | |**Bus System Interface Type**|[PCI Express](https://www.wikidata.org/wiki/Q206924) x1| 36 | |**External Connectors**|[68-pin VHDCI](https://www.wikidata.org/wiki/Q4052507)| 37 | |**Data Transfer Rate**|Up to 320 MByte/sec.| 38 | 39 | #### Note on bracket height 40 | 41 | Note that by default the controller has a standard height (120 mm) bracket that won't fit into a low-profile (79.2 mm) slot! When these controllers were sold new, they came with a replacement low-profile bracket, but these are often not included with used ones. The low-profile replacement brackets are sometimes sold separately on eBay. 42 | 43 | ### Connecting / disconnecting the tape reader 44 | 45 | Always make sure that both the tape reader and the computer are switched off before connecting or disconnecting. Not doing so may cause damage to the computer, the tape reader or both. 46 | 47 | ## Cleaning cartridges 48 | 49 | The drive needs to be cleaned periodically with a cleaning cartridge. 50 | 51 | ## Software 52 | 53 | [tapeimgr](https://github.com/KBNLresearch/tapeimgr), running in [BitCurator](https://bitcurator.net/). 54 | 55 | ## Procedure for reading a tape 56 | 57 | 1. Check the write-protect tab on the bottom of the tape, and slide it to 58 | the open position: 59 | 60 | ![](./img/dds-protect.jpg) 61 | 62 | 2. Insert the tape into the drive. Make sure the printed side is on top, and that the tape is inserted in the direction of the arrow symbol: 63 | 64 | ![](./img/dds-insert.jpg) 65 | 66 | 3. Create an empty directory for storing the extracted tape contents. 67 | 68 | 4. Start *tapeimgr* from the OS's main menu (in Ubuntu 18.04 the *tapeimgr* item is located under *System Tools*), or by clicking the *tapeimgr* shortcut on the desktop. 69 | 70 | ![](./img/tapeimgr-1.png) 71 | 72 | 5. Use the *Select Output Directory* button to navigate to the empty directory created under step 3. Click on the *UUID* button to generate a unique identifier. Use the *Description* field to enter a description of the tape (e.g. the title that is written on its label or inlay card). Optionally, use the *Notes* field to record anything else worth mentioning about the tape (e.g. if the *Description* is ambiguous because the writing on the tape's label is illegible, make a note of it here). 73 | 74 | 6. Press the *Start* button to start the extraction, and then wait for *tapeimgr* to finish (depending on the amount of data on the tape this may take several hours). You can monitor the progress of the extraction procedure in the progress window: 75 | 76 | ![](./img/tapeimgr-2.png) 77 | 78 | Note that the screen output is also written to a log file in the output directory. A prompt appears when the extraction has finished: 79 | 80 | ![](./img/tapeimgr-success.png) 81 | 82 | If the extraction finished without any errors, the output directory now contains the following files: 83 | 84 | ![](./img/tapeimgr-files.png) 85 | 86 | Here, **file000001.dd** through **file000003.dd** are the extracted files; **checksums.sha512** contains the SHA512 checksums of the extracted files, **metadata.json** contains some basic metadata and **tapeimgr.log** is the log file. 87 | 88 | 7. The tape is automatically ejected when *tapeimgr* has finished. All done! 89 | 90 | ## Cleaning the tape drive 91 | 92 | If the orange "clean" indicator is lit, it is time to clean the drive: 93 | 94 | Steps: 95 | 96 | 1. Insert a cleaning cartridge into the drive and wait for the clean to take place. The tape is ejected at the end. 97 | 98 | In order to avoid read errors, it is important to *not* continue reading tapes until the drive has been cleaned. 99 | -------------------------------------------------------------------------------- /doc/floppy-35.md: -------------------------------------------------------------------------------- 1 | # 3.5" floppy disk 2 | 3 | ## Introduction 4 | 5 | [3.5" floppy disk](https://en.wikipedia.org/wiki/Floppy_disk#%E2%80%8B3_1%E2%81%842-inch_floppy_disk). 6 | 7 | ## Hardware 8 | 9 | ### Floppy disk drive 10 | 11 | |**Model**|[BaseTech USB Floppy Disk Drive refurbished](https://web.archive.org/web/20181008141513/http://www.produktinfo.conrad.com/datenblaetter/1100000-1199999/001170561-an-01-ml-BASETECH_FLOPPY_LAUFWERK_USB_de_en_fr_nl.pdf)| 12 | |:--|:--| 13 | |**Connection**|USB 2.0/1.1| 14 | |**Type of floppy disk**|High Density (HD) 1.44 MByte, Double Density (DD), 720 kByte| 15 | 16 | 30 | 31 | ## Note on write protection 32 | 33 | An earlier version of this document recommended the use of a forensic write blocker (Tableau Forensic USB 3.0 Bridge). More recent experience has shown that this particular write blocker (and possibly other models as well) does not work well with USB floppy drives, and may result in faulty or incomplete captures. We have received confirmation of this from the manufacturer. For example, when the write blocker is first used to image a Double Density (720 kB) floppy, followed by a High Density (1.44 MB) one, the High Density image only contains the first 720 kB of the data on the floppy! 34 | 35 | Because of this we discourage the use of write blockers for 3.5" floppies, and suggest to always use the following write-protection measures instead: 36 | 37 | 1. Disable automatic mounting of removable media at the Operating System level. This is already the default in BitCurator. For other common Ubuntu-derived Linux distros, [follow the instructions here](./disable-automount.md). 38 | 39 | 2. Always set the floppy's square write-protection tab to the "protected" position. 40 | 41 | ## Software 42 | 43 | [diskimgr](https://github.com/KBNLresearch/diskimgr), running in [BitCurator](https://bitcurator.net/). 44 | 45 | ## Procedure for creating a disk image 46 | 47 | 1. Connect the floppy disk drive to the right-hand USB connector on the write blocker. 48 | 49 | 2. Make sure the floppy's write-protection tab is set to the "protected" position (the hole in the corner of the disk must be uncovered): 50 | 51 | ![](./img/write-protection.png) 52 | 53 | 3. Start *diskimgr* from the OS's main menu (in Ubuntu 18.04 the *diskimgr* item is located under *System Tools*). 54 | 55 | ![](./img/diskimgr-1.png) 56 | 57 | 4. Use the *Select Output Directory* button to navigate to an empty directory where the disk image and its associated metadata will be stored (or create a new directory from the dialog). 58 | 59 | 5. Click on the *UUID* button to generate a unique identifier. 60 | 61 | 6. Change the *Prefix* value to the name you want to give to the image file, and use the *Description* field to enter a description of the floppy (e.g. the title or description that is written on its label). Optionally, use the *Notes* field to record anything else worth mentioning about the floppy (e.g. if the *Description* is ambiguous because the writing on label is illegible, make a note of it here). 62 | 63 | 7. Insert the floppy into the drive, and press *diskimgr*'s *Refresh* button. 64 | 65 | 8. Now select the floppy from the *Block device* drop-down menu (you can usually identify it by its size, which is 1.4 MiB for a High-Density disc, and 720 KiB for a Double-Density disc). 66 | 67 | 9. Press the *Start* button to start imaging, and then wait for *diskimgr* to finish. You can monitor the progress of the imaging procedure in the progress window: 68 | 69 | ![](./img/diskimgr-2.png) 70 | 71 | Note that the screen output is also written to a log file in the output directory. If the imaging finished without any errors, the following prompt appears: 72 | 73 | ![](./img/diskimgr-success.png) 74 | 75 | The output directory now contains the following files: 76 | 77 | ![](./img/diskimgr-files.png) 78 | 79 | Here, **ks.img** is the ISO image; **checksums.sha512** contains the SHA512 checksum of the image file, **metadata.json** contains some basic metadata and **diskimgr.log** is the log file. 80 | 81 | 10. If the imaging procedure *did* result in any errors, follow the *Additional steps in case of errors* section below. Otherwise, take out the floppy. 82 | 83 | 84 | ## Additional steps in case of errors 85 | 86 | If the reading of the floppy with *dd* resulted in errors, the following prompt will appear: 87 | 88 | ![](./img/error-dd.png) 89 | 90 | If this happens, follow these steps: 91 | 92 | 1. Click *Yes*. *Diskimgr* will now move the current image file to a subdirectory *dd-failed*, and retry imaging the disc with the *ddrescue* tool (which is often better at recovering data from defective media). 93 | 94 | 2. If *diskimgr* still reports errors after the first pass with *ddrescue*, you can run additional passes to improve the result. Try re-running it in *Direct disc* mode (which can be selected from *diskimgr*'s interface). Another useful technique is to run additional *ddrescue* passes with a different floppy drive. 95 | 96 | 97 | 98 | 3. Take the floppy out of the drive after the last *ddrescue* pass. All done! 99 | 100 | ## Interrupting ddrescue 101 | 102 | Running imaging processes can be stopped by pressing the *Interrupt* button. This is particularly useful for *ddrescue* runs, which may require a long time for floppies that are badly damaged. Note that interrupting *ddrescue* will not result in any data loss, and interrupted runs can be resumed at a later time (see below). Interrupting *dd* is not advised. 103 | 104 | ## Resuming an interrupted ddrescue run 105 | 106 | Follow these steps to resume a *ddrescue* run that was previously interrupted: 107 | 108 | 1. After launching *diskimgr*, set the output directory to the directory of the interrupted run. 109 | 110 | 2. Set **Read method** to *ddrescue*. 111 | 112 | 3. Click on the **Load existing metadata** button; this loads the previously entered *Prefix*, *Extension*, *Identifier*, *Description* and *Notes* values. 113 | 114 | 4. Hit the **Start** button. Now *ddrescue* will simply pick up on where the interrupted run stopped. 115 | 116 | ## Remarks 117 | 118 | 119 | 120 | Using a write blocker may be overkill if we also use the write protection tabs on the floppies? (Then again, better safe than sorry.) -------------------------------------------------------------------------------- /doc/rough-notes/tape-notes.md: -------------------------------------------------------------------------------- 1 | # Tape notes 2 | 3 | Rough working notes, to be sorted/rearranged later. 4 | 5 | ## Software 6 | 7 | Additional software for working with tape devices: 8 | 9 | sudo apt install lsscsi 10 | 11 | Software for translating Microsoft NTBackup stream (MTF) to TAR: 12 | 13 | 14 | 15 | Download above TAR, extract to directory, go to directory and then build using: 16 | 17 | make 18 | 19 | ## General tape commands 20 | 21 | List installed SCSI tape devices: 22 | 23 | lsscsi 24 | 25 | Result: 26 | 27 | [0:0:2:0] tape HP C1533A A708 /dev/st0 28 | [1:0:0:0] disk ATA WDC WD2500AAKX-6 1H18 /dev/sda 29 | [3:0:0:0] cd/dvd hp DVD A DH16ABSH YHDD /dev/sr0 30 | [7:0:0:0] disk WD Elements 25A2 1021 /dev/sdb 31 | 32 | So our tape device is `/dev/st0`. For reading however we will use the " non-rewind" tape device which is `/dev/nst0`. The difference betweeen these devices: 33 | 34 | - When using `/dev/st0`, the tape is automatically rewound after each read/write operation (e.g. using *dd*). 35 | - When using `/dev/nst0`, the tape is left at its current position after each read/write operation. 36 | 37 | The [*mt*](https://linux.die.net/man/1/mt) command is used for all tape operations. It must be run as root (so use *sudo*). 38 | 39 | Display tape status: 40 | 41 | sudo mt -f /dev/st0 status 42 | 43 | Result: 44 | 45 | drive type = 114 46 | drive status = 318767104 47 | sense key error = 0 48 | residue count = 0 49 | file number = 0 50 | block number = 0 51 | 52 | Rewind tape: 53 | 54 | sudo mt -f /dev/st0 rewind 55 | 56 | Eject tape: 57 | 58 | sudo mt -f /dev/st0 eject 59 | 60 | ## Note on dd usage 61 | 62 | From [forensicswiki](https://www.forensicswiki.org/wiki/Dd): 63 | 64 | > Having a bigger blocksize is more efficient, but if you use a 1MB block as an example and have a read error in the first sector, then dd will null fill the entire MB. Thus you should use as small a blocksize as feasible. 65 | > 66 | > But with linux if you go below 4KB blocksize, you can hit really bad performance issues. It can be as much as 10x slower to use the default 512 byte block as it is to use a 4KB block. 67 | > 68 | > Without noerror and sync, you basically don't have a forensic image. For forensic images they are mandatory. 69 | 70 | > dd by itself does not hash, that is why the alternate command is provided. 71 | 72 | See also tape-specific comments in *Cautions* section! 73 | 74 | ## Make test tape 75 | 76 | Do a short erase: 77 | 78 | sudo mt -f /dev/st0 erase 1 79 | 80 | 81 | **NOTE** don 't do an erase (not even a short one) because it takes forever and the only way to stop it is a full system reboot! 82 | 83 | Write two sessions: 84 | 85 | sudo tar -cvf /dev/nst0 /home/bcadmin/jpylyzer-test-files 86 | sudo tar -cvf /dev/nst0 /home/bcadmin/forensicImagingResources 87 | sudo tar -cvf /dev/nst0 /media/bcadmin/Elements/testBitCurator/testfloppy 88 | 89 | Extract: 90 | 91 | sudo dd if=/dev/nst0 of=session1conv.dd bs=16384 conv=noerror,sync 92 | 93 | Result: 46.5 MB file. When unpacking as tar the archives are incomplete and/or not readable! 94 | 95 | Second attempt, omitting the *conv* swich: 96 | 97 | sudo dd if=/dev/nst0 of=session1conv.dd bs=16384 98 | 99 | Result: 29.1 MB file. Unpacking as tar works! 100 | 101 | From the [dd documentation](http://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html): 102 | 103 | > sync 104 | > Pad every input block to the size of the ibs= buffer, appending null bytes. (If either block or unblock is also specified, append characters, rather than null bytes.) 105 | 106 | A comparison of the 2 extracted files in a hex editor shows a block of around 6000 null bytes are inserted around offset 10240, adding about 6000 bytes. So let' s try the extraction with a block sixe of 10240 bytes: 107 | 108 | sudo dd if=/dev/nst0 of=session1convbs10240.dd bs=10240 conv=noerror,sync 109 | 110 | Result: produces valid TAR archive of 29.1 MB. From the [tar docs](https://www.gnu.org/software/tar/manual/html_node/Blocking.html): 111 | 112 | > In a standard tar file (no options), the block size is 512 and the record size is 10240, for a blocking factor of 20. 113 | 114 | Solution: estimate block size by successively adding 512 bytes to start value. 115 | 116 | TODO: 117 | 118 | - What if the actual block size is SMALLER than 4096 bytes (current start value)? Would assume that this would result in addition of padding bytes. 119 | 120 | - What if the block sizes varie across tape sessions? 121 | 122 | Try with ddrescue: 123 | 124 | sudo ddrescue -b 10240 -v /dev/nst0 session1.dd session1.log 125 | 126 | ## Block / record size tests 127 | 128 | Write with 1024 byte record size: 129 | 130 | sudo tar -cvf /dev/nst0 -b2 /media/bcadmin/Elements/testBitCurator/testfloppy 131 | 132 | Write with 4096 byte record size: 133 | 134 | sudo tar -cvf /dev/nst0 -b8 /media/bcadmin/Elements/testBitCurator/testfloppy 135 | 136 | Write with 8192 byte record size: 137 | 138 | sudo tar -cvf /dev/nst0 -b16 /media/bcadmin/Elements/testBitCurator/testfloppy 139 | 140 | ## Procedure for reading an NTBackup tape 141 | 142 | 1. Load the tape 143 | 144 | 2. Determine the block size by entering: 145 | 146 | sudo dd if=/dev/st0 of=tmp.dd ibs=128 count=1 147 | 148 | If this results in a *Cannot allocate memory* error message, repeat the above command with a larger ibs value (e.g. 256). Repeat until the error goes away and some data is read. For instance: 149 | 150 | sudo dd if=/dev/st0 of=tmp.dd ibs=512 count=1 151 | 152 | Results in: 153 | 154 | 1+0 records in 155 | 1+0 records out 156 | 512 bytes copied, 0.308845 s, 1.7 kB/s 157 | 158 | Which means that the block size is 512 bytes. 159 | 160 | An alternative method is described [here](https://www.linuxquestions.org/questions/linux-general-1/reading-%27unknown%27-data-from-a-tape-4175500596/#post5147408): 161 | 162 | > Easiest way to find the actual block size for a given file on the tape is to run 163 | > 164 | > `dd if=/dev/nst0 of=/dev/null bs=64k count=1` 165 | > 166 | > and look at the number of bytes dd reports for that single block. 167 | > 168 | > Most basic way to compare: 169 | > 170 | > 171 | > `cmp <(dd if=/dev/nst0 bs=32k) <(dd if=/dev/nst1 bs=32k) && echo OK` 172 | > 173 | > Adjust the block size as you wish, of course, as long as it is large enough. 174 | 175 | 3. Read blocks (note that we're using the non-rewinding tape device ` /dev/nst0` here): 176 | 177 | for f in `seq 1 10`; do sudo dd if=/dev/nst0 of=tapeblock`printf "%06g" $f`.bin ibs=512; done 178 | 179 | Output: 180 | 181 | 2251822+0 records in 182 | 2251822+0 records out 183 | 1152932864 bytes (1.2 GB, 1.1 GiB) copied, 5253.08 s, 219 kB/s 184 | 1667+0 records in 185 | 1667+0 records out 186 | 853504 bytes (854 kB, 834 KiB) copied, 3.23535 s, 264 kB/s 187 | 0+0 records in 188 | 0+0 records out 189 | 0 bytes copied, 0.0167298 s, 0.0 kB/s 190 | dd: error reading '/dev/nst0': Input/output error 191 | 0+0 records in 192 | 0+0 records out 193 | 0 bytes copied, 0.00017777 s, 0.0 kB/s 194 | 195 | Question: why 10 iterations? What does each iteration represent (a backup session? something else?) 196 | 197 | 4. Rewind the tape: 198 | 199 | sudo mt -f /dev/st0 rewind 200 | 201 | 5. Eject the tape: 202 | 203 | sudo mt -f /dev/st0 eject 204 | 205 | ## Processing the extracted files 206 | 207 | 1. Join extracted files together using something like this: 208 | 209 | cat tapeblock000001.bin tapeblock000002.bin > tape.bin 210 | 211 | ## Cleaning cartridges 212 | 213 | - [HP Cleaning Cartridge DDS for use with SureStore and all other DDS drives](https://www.bol.com/nl/p/hp-cleaning-cartridge-dds-for-use-with-surestore-and-all-other-dds-drives/9200000008139219/) 214 | 215 | - [Fujifilm DDS Cleaning Tape](https://www.fujifilm.eu/nl/producten/opnamemedia/data-storage-media/p/dds-cleaning-tape) 216 | 217 | ## DDS-3 drive 218 | 219 | Model: IBM STD224000N (internal drive). Cannot find any tech specs on it. 220 | 221 | When it is connected to the machine, on bootup it shows: 222 | 223 | Time-out failure during SCSI inquiry command 224 | 225 | Then after a few more tries the screen goes blank and the machine hangs. 226 | 227 | 228 | From [SCSI controller docs](http://download.adaptec.com/pdfs/installation_guides/scsi_installation_and_user_guide_02_07.pdf): 229 | 230 | >- Internal Ultra320, Ultra160, and Ultra2 SCSI LVD devices come from the factory with termination disabled and cannot be changed. Proper termination for these internal devices is provided by the built-in terminator at the end of the 68-pin internal LVD SCSI cable. 231 | >- Termination on SE internal SCSI devices is usually controlled by manually setting a jumper or a switch on the device, or by physically removing or installing one or more resistor modules on the device. 232 | 233 | So could be a termination issue ... 234 | 235 | UPDATE: it seems this [IBM 7206-110](http://www.dich.com.tw/Product/Storage/Tape/IBM/7206%20Model%20110%2012%20GB%20External%204mm%20DDS-3%20Tape%20Drive.pdf) external drive is basically the external variant of this drive. See also [here](ftp://ftp.software.ibm.com/software/server/firmware/12GB4mm.htm). -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /doc/df-resources.md: -------------------------------------------------------------------------------- 1 | # Digital forensics and web archaeology resources 2 | 3 | ## Tapes 4 | 5 | - [Recovering '90s Data Tapes - Experiences From the KB Web Archaeology project (iPres 2019 paper)](https://www.bitsgalore.org/2019/09/09/recovering-90s-data-tapes-experiences-kb-web-archaeology) 6 | - [Tape Driver Semantics](https://wiki.zmanda.com/index.php/Tape_Driver_Semantics) - Excellent summary of the behavior of tapes on UNIX systems. 7 | - [How to use the DAT-tape with Linux](http://www.cs.inf.ethz.ch/stricker/lab/linux_tape.html) 8 | - [HP Surestore and StorageWorks DAT - HP DAT Accessories and Part Numbers](https://web.archive.org/web/20181101135618/https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-lpg50205) 9 | - [HP StorageWorks DDS/DAT Media - DDS/DAT Media Compatibility Matrix](https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-lpg50457) (note: not accessible anymore, and no usable snapshots in web archives!) 10 | - [HP DAT Compatibility Chart](https://web.archive.org/web/20240315125731/https://www.artisantg.com/info/HP_DAT_Compatability_Chart.pdf) 11 | - [15 Useful Linux and Unix Tape Managements Commands For Sysadmins](https://www.cyberciti.biz/hardware/unix-linux-basic-tape-management-commands/) 12 | - [Linux Set the Block Size for a SCSI Tape Device](https://www.cyberciti.biz/faq/rhel-centos-debian-set-tape-blocksize/) 13 | - [Reading VMS tapes from Linux](https://www.tldp.org/HOWTO/VMS-to-Linux-HOWTO/x838.html) 14 | - [Reading Tapes Written on Other Systems](http://www.astro.sunysb.edu/sysman/fits.html) 15 | - [reading 'unknown' data from a tape](https://www.linuxquestions.org/questions/linux-general-1/reading-%27unknown%27-data-from-a-tape-4175500596/) 16 | - [Copying Files and File Systems to Tape (Oracle documentation)](https://docs.oracle.com/cd/E19455-01/805-7228/6j6q7uf24/index.html) - explains pax, tar and cpio 17 | - [How to copy data from DDS tape to UNIX station](https://community.hpe.com/t5/System-Administration/How-to-copy-data-from-DDS-tape-to-UNIX-station/td-p/4780851#.W9MEpxCxU3E) 18 | - [Purpose of ibs/obs/bs in dd](https://stackoverflow.com/questions/1354938/purpose-of-ibs-obs-bs-in-dd) - one of the answers explains particular importance of block size in case of tapes 19 | - [Duplicating a tape drive using dd](https://www.linuxquestions.org/questions/linux-newbie-8/duplicating-a-tape-drive-using-dd-4175592839/) - more info on block size. 20 | - [What does dd conv=sync,noerror do?](https://superuser.com/questions/622541/what-does-dd-conv-sync-noerror-do) - confirms that using these flags is generally a bad idea 21 | - [Write protect in DLT IV Data cartridge](https://web.archive.org/web/20181031154114/https://community.hpe.com/t5/StoreEver-Tape-Storage/Write-protect-in-DLT-IV-Data-cartridge/td-p/129718https://community.hpe.com/t5/StoreEver-Tape-Storage/Write-protect-in-DLT-IV-Data-cartridge/td-p/129718) 22 | - [mhVTL Virtual Tape Library](https://www.mhvtl.com/); source code [here](https://github.com/markh794/mhvtl) 23 | - [mhVTL install script for Ubuntu](https://gist.github.com/hrchu/3eb1c0aa9994df0328037fff04cd889d) 24 | - [The Source of All Tape Knowledge](http://www.subspacefield.org/~vax/unix_tape.html) 25 | - [Tapeimgr](https://github.com/KBNLresearch/tapeimgr) - simple format-agnostic tape imaging / extraction software (GUI application, wraps around dd/mt) 26 | - [Experiences of an LTO/LTFS beginner](https://digitensions.home.blog/2019/01/15/technologic/) 27 | - [Data by the Foot](https://campuspress.yale.edu/borndigital/2019/08/30/data-by-the-foot/) - blog about tape data recovery at Yale University Library 28 | 29 | ## SCSI 30 | 31 | - [What kind of SCSI do I have? Single-ended or differential SCSI interface?](http://www.paralan.com/sediff.html) -*really* helpful for identifying SCSI connector types! 32 | - [LVD, SE, HVD, SCSI Compatibility - Or Not](http://www.paralan.com/scsiexpert.html) 33 | - [SCSI articles on Paralan (AKA the SCSI Bible)](http://www.paralan.com/aboutscsi.html) 34 | - [SCSI Connector Guide](https://www.cablestogo.com/learning/connector-guides/scsi) 35 | - [Adaptec SCSI controller specifications](https://storage.microsemi.com/en-us/support/scsi/) 36 | - [SCSI Termination](https://support.hpe.com/hpsc/doc/public/display?docId=tis14318) 37 | - Here's a good series of articles on pctechguide.com: 38 | - [SCSI Adapters](https://www.pctechguide.com/how-to-install-a-scsi-device/scsi-adapters) 39 | - [SCSI Internal Intro](https://www.pctechguide.com/how-to-install-a-scsi-device/scsi-internal-intro) 40 | - [SCSI Internal Configuration](https://www.pctechguide.com/how-to-install-a-scsi-device/scsi-internal-configuration) 41 | - [SCSI Internal Mounting](https://www.pctechguide.com/how-to-install-a-scsi-device/scsi-internal-mounting) 42 | - [SCSI Internal Connections](https://www.pctechguide.com/how-to-install-a-scsi-device/scsi-internal-connections) 43 | - [SCSI Internal Software](https://www.pctechguide.com/how-to-install-a-scsi-device/scsi-internal-software) 44 | - [SCSI External](https://www.pctechguide.com/how-to-install-a-scsi-device/scsi-external) 45 | - [Installing a SCSI device – making the connections](https://www.pctechguide.com/how-to-install-a-scsi-device/installing-a-scsi-device-making-the-connections) 46 | - [SCSI External Config](https://www.pctechguide.com/how-to-install-a-scsi-device/scsi-external-config) 47 | - [SCSI External Software](https://www.pctechguide.com/how-to-install-a-scsi-device/scsi-external-software) 48 | - [Squishy: a platform for working with old SCSI devices with modern systems in a flexible manner](https://github.com/lethalbit/squishy) 49 | 50 | ## Tape archive formats (and how to read / extract them) 51 | 52 | - [Unix dump format](http://fileformats.archiveteam.org/wiki/Unix_dump) 53 | - [dump(8)](https://linux.die.net/man/8/dump) 54 | - [restore(8)](https://linux.die.net/man/8/restore) 55 | - [restore Linux Commands](https://www.hscripts.com/tutorials/linux-commands/restore.html) 56 | - [How to Restore UFS Files Interactively](https://docs.oracle.com/cd/E19253-01/817-5093/bkuprestoretasks-63510/index.html) (excellent resource for restoring dump files) 57 | - [Microsoft™ Tape Format Specification Version 1.00a](http://laytongraphics.com/mtf/MTF_100a.PDF) 58 | - [mtftar](https://github.com/sjmurdoch/mtftar) - mtftar is a tool for translating a Microsoft NTBackup (MTF) stream to a TAR stream 59 | - [Recovering NTBackup Tapes](https://www.108.bz/posts/it/recovering-ntbackup-tapes/) 60 | 61 | ## Floppy disks 62 | 63 | - [Rescuing Floppy Disks (ArchiveTeam)](https://www.archiveteam.org/index.php/Rescuing_Floppy_Disks) 64 | - [The Archivist’s Guide to KryoFlux](https://github.com/archivistsguidetokryoflux/archivists-guide-to-kryoflux) 65 | - [KryoFlux floppy case](https://www.thingiverse.com/thing:3089895) 66 | - [Accessing Mac Formatted Floppy Disks without a Kryoflux](https://porterolsen.wordpress.com/2016/06/15/accessing-mac-formatted-floppy-disks-without-the-kryoflux/) 67 | - [Getting Data Out Of Its Floppy Cage](http://www.wcsarchivesblog.org/getting-data-out-of-its-floppy-cage/) - provides some details on how to get the FC5025 floppy controller working with BitCurator 68 | - [Use Guide for the FC5025 Floppy Disk Controller](https://web.archive.org/web/20180507194729/https://mith.umd.edu/vintage-computers/fc5025-operation-instructions) 69 | - [Bodging a case for a 5.25″ floppy drive](https://radd.dsalo.info/2017/01/bodging-a-case-for-a-5-25-floppy-drive/) 70 | - [Digital Archaeology and/or Forensics: Working with Floppy Disks from the 1980s](https://journal.code4lib.org/articles/11986) 71 | - [Adafruit Floppy](https://github.com/adafruit/Adafruit_Floppy) - a helper library to abstract away interfacing with floppy disk drives in a cross-platform and open source library. 72 | - [A Dogged Pursuit: Capturing Forensic Images of 3.5” Floppy Disks](https://practicaltechnologyforarchives.org/issue2_waugh/) 73 | - [Emulating the early Macintosh floppy drive](https://thomasw.dev/post/mac-floppy-emu/) - in-depth post about low-level aspects of vintage Macintosh floppies. 74 | - [A Guide to Imaging Obscure Floppy Disk Formats](https://zenodo.org/doi/10.5281/zenodo.13766982) 75 | 76 | ## Optical media 77 | 78 | - [An Introduction to Optical Media Preservation (Alexander Duryee)](https://journal.code4lib.org/articles/9581) 79 | - [Developing a Robust Migration Workflow for Preserving and Curating Hand-held Media](https://arxiv.org/abs/1309.4932) 80 | - [Preserving Write-Once DVDs (Blood Report for LoC)](http://www.digitizationguidelines.gov/audio-visual/documents/Preserve_DVDs_BloodReport_20140901.pdf) 81 | - [Assessing High-volume Transfers from Optical Media at NYPL](https://journal.code4lib.org/articles/15908) 82 | - [CD Formats and Their Longevity - FAQ (Canadian Conservation Institute)](https://web.archive.org/web/20170825093105if_/http://canada.pch.gc.ca/eng/1456339921100) 83 | - [Andy McFadden's CD-Recordable FAQ](https://www.cdrfaq.org/) 84 | - [Preserving optical media from the command-line](https://www.bitsgalore.org/2015/11/13/preserving-optical-media-from-the-command-line) 85 | - [A simple workflow tool for imaging optical media using readom and ddrescue](https://www.bitsgalore.org/2019/03/22/a-simple-workflow-tool-for-imaging-optical-media-using-readom-and-ddrescue) 86 | - [Imaging CD-Extra / Blue Book discs](https://www.bitsgalore.org/2017/04/25/imaging-cd-extra-blue-book-discs) 87 | - [Image and Rip Optical Media Like A Boss!](https://www.bitsgalore.org/2017/06/19/image-and-rip-optical-media-like-a-boss) 88 | - [Isolyzer](https://github.com/KBNLresearch/isolyzer) - tool that verifies if the file size of a CD / DVD image ("ISO image") is consistent with the information in its filesystem-level headers. 89 | - [Acronova Nimbie Drive Replacement Instruction](https://web.archive.org/web/20210915163606/http://www.usarcades.com/wp-content/uploads/Drive-Replacement-Instruction-v1.2.pdf) 90 | - [To Everything There Is a Session: A Time to Listen, a Time to Read Multi-session CDs (Dietrich, Nelson)](https://journal.code4lib.org/articles/17208) 91 | - [The Forgotten Disc: Synthesis and Recommendations for Viable VCD Preservation (Weaver, Blewer)](https://journal.code4lib.org/articles/17406) 92 | - [The Working Archivist's Guide to Enthusiast CD-ROM Archiving Tools (De Meo)](https://www.mistys-internet.website/blog/blog/2024/09/13/the-working-archivists-guide-to-enthusiast-cd-rom-archiving-tools/) 93 | - [What You Might Miss When Backing Up CDs (De Meo)](https://www.mistys-internet.website/blog/blog/2025/01/23/what-you-might-miss-when-backing-up-cds/) 94 | 95 | ## Cables 96 | 97 | - [The Cable Bible](https://amiaopensource.github.io/cable-bible/) 98 | 99 | ## PCI Cards 100 | 101 | - [PCI card types](https://upload.wikimedia.org/wikipedia/commons/1/15/PCI_Keying.svg) 102 | - [Startech PCI Express to PCI Adapter Card](https://www.startech.com/nl/en/Cards-Adapters/Slot-Extension/PCI-Express-to-PCI-Adapter-Card~PEX1PCI1) 103 | - [Sintech PCI-E Express X1 to Dual PCI Riser Extender Card](https://www.amazon.com/gp/product/B00KZHDSLQ?psc=1&redirect=true&ref_=oh_aui_detailpage_o07_s00) 104 | 105 | ## Web archaeology 106 | 107 | - [DIY Handboek voor Webarcheologie](https://hart.amsterdam/image/2017/11/17/20171116_freeze_diy_handboek.pdf) 108 | - [Project "The Digital City Revives" A Case Study of Web Archaeology](https://hart.amsterdam/image/2016/11/28/20160730_redds_tjardadehaan.pdf) 109 | - [Server-side Preservation of Dynamic Websites](http://publications.beeldengeluid.nl/pub/633/) 110 | -[Archaeology of the Amsterdam digital city; why digital data are dynamic and should be treated accordingly](https://www.tandfonline.com/doi/full/10.1080/24701475.2017.1309852) 111 | 112 | ## Data rescue and digital forensics (general) 113 | 114 | - [Digital Forensics and Preservation (DPC Report)](http://dx.doi.org/10.7207/twr12-03) 115 | - [Digital Archaeology: Rescuing Neglected and Damaged Data Resources](http://www.ukoln.ac.uk/services/elib/papers/supporting/pdf/p2.pdf) 116 | - [Getting to Know FRED: Introducing Workflows for Born-Digital Content](https://practicaltechnologyforarchives.org/issue4_prael_wickner/) 117 | - [Digital Archives Workstation Update: KryoFlux, FRED, and BitCurator Walk into a Bar…](https://blogs.princeton.edu/techsvs/2017/10/03/digital-archives-workstation-update-kryoflux-fred-and-bitcurator-walk-into-a-bar/) 118 | - [NYPL’s New Digital Archives Lab](https://www.nypl.org/blog/2017/01/11/nypls-new-digital-archives-lab) 119 | - [Michigan University Digital Preservation Lab 2.0](https://www.lib.umich.edu/blogs/bits-and-pieces/digital-preservation-lab-20) 120 | - [Building Audio, Video, And Data-Rescue Kits](https://radd.dsalo.info/wp-content/uploads/2017/10/BuildDocumentation.pdf) 121 | - [Disk Image Formats](https://wiki.harvard.edu/confluence/display/digitalpreservation/Disk+Image+Formats) 122 | - [Automated Processing of Disk Images and Directories in BitCurator (Tessa Walsh)](https://www.bitarchivist.net/blog/2017-05-01-buf2017/) 123 | - [Challenges of Dumping/Imaging old IDE Disks](https://openpreservation.org/blog/2013/03/20/challenges-dumpingimaging-old-ide-disks/) 124 | - [BitCurator Workflows](https://bitcuratorconsortium.org/workflows) 125 | - [Museum of Obsolete Media](https://obsoletemedia.org/) 126 | - [Born Digital Workflows, UNC at Chapel Hill Libraries, Wilson Special Collections Library](http://wilsonborndigital.web.unc.edu/) 127 | - [CCA Digital Archives Processing Manual](https://github.com/CCA-Public/digital-archives-manual) - includes LOTS of detailed workflow descriptions 128 | - [Guide to identifying obsolete digital media](https://www.projectcest.be/wiki/Publicatie:Handleiding_Verouderde_Dragers_Herkennen) (in Dutch) 129 | - [Digital Repair Cafe](https://automatic-ingest-digital-archives.github.io/Digital-Repair-Cafe/) - links to various capture workflows. 130 | - [How to Read a Floppy Disk on a Modern PC or Mac](https://www.howtogeek.com/669331/how-to-read-a-floppy-disk-on-a-modern-pc-or-mac/) 131 | - [Disk Imaging Decision Factors](https://dannng.github.io/disk-imaging-decision-factors.html) 132 | - [Imaging Digital Media for Preservation with LAMMP](https://resources.culturalheritage.org/emg-review/volume-three-2013-2014/mckinley/) 133 | - [Towards Best Practices In Disk Imaging: A Cross-Institutional Approach](https://resources.culturalheritage.org/emg-review/volume-6-2019-2020/colloton/) 134 | - [NMAAHC Disk Imaging Workshop](http://eddycolloton.com/blog/2022/4/6/nmaahc-disk-imaging-workshop) 135 | - [National Institute of Standards and Technology (NIST) Computer Forensic Tool Testing (CFTT) Reports](https://www.dhs.gov/science-and-technology/nist-cftt-reports) 136 | - [Cambridge University Library Transfer Service](https://digitalpreservation-blog.lib.cam.ac.uk/the-transfer-service-40bddeff0d32) --------------------------------------------------------------------------------