├── README.md ├── TESTING.md └── regression.sh /README.md: -------------------------------------------------------------------------------- 1 | # Crossmeta BSD POSIX for Windows 2 | ![crossmeta](https://www.crossmeta.io/wp-content/themes/allegiant/images/crossmeta-fs.jpg) 3 | 4 | Crossmeta provides BSD style POSIX file system operations to Windows programs using standard NT kernel interfaces. In addition, it has support for fork/wait for process management, and more potential to offer other Unix-like environment. It has ash shell (Almquist shell) plus tiny versions of many common UNIX utilities, sed, awk, grep, and diff etc. Crossmeta provides all these within the standard Win32 subsystem in Windows NT, without segregation. 5 | 6 | ## Crossmeta Advantage 7 | #### Dynamic kernel modules 8 | Crossmeta is fully functional dynamic loadable module, that can be loaded on demand. And when not used, the complete kernel drivers can be unloaded. 9 | Just like any other dynamic loadable module, Crossmeta cannot be unloaded when applications have active references. 10 | 11 | #### Not a separate subsystem in Windows 12 | Crossmeta runs within WIN32 subsystem and supports WIN32/WIN64 binaries. 13 | 14 | #### fork and wait support 15 | Crossmeta has basic support for fork, which is efficient light-weight fork and 16 | is Copy-On-Write as in other POSIX systems. 17 | The wait() syscall can be used to harvest return status of forked processes. 18 | 19 | #### Unlink of open files 20 | Just like any other POSIX system, Crossmeta supports deletion and rename of 21 | open files. 22 | Only when the reference to unlinked inode becomes zero, the file gets deleted automatically. 23 | 24 | #### POSIX file system operations 25 | Since Crossmeta includes BSD VFS it supports all the standard 26 | chown, chmod, hardlinks, symbolic links, and rename, etc VFS calls in its own VFS namespace. 27 | 28 | #### Mounting and Unmoung File systems 29 | With mount/unmount on a directory the file system is best visualized as a tree, rooted as it were, at /. /dev, /usr, /home, and the other directories in the root directory are branches, which may have their own branches, such as /usr/local, and so on. 30 | 31 | #### Support for file systems 32 | Crossmeta has impressive support for following file systems 33 | 34 | | Filesystem | rw | Journal | 35 | | ---------- | ----- | ----- | 36 | | XFS | Read/Write | Journnal (Log ver2, No support for delay-log) | 37 | | EXT2 | Read/Write | None | 38 | | EXT3 | Read/Write | No Journal | 39 | | EXT4 | Read/Write | No Journal | 40 | | ReiserFs | Read | - | 41 | | Apple HFS+ | Read| No Journal | 42 | 43 | #### FUSE User mode file system 44 | Filesystem in Userspace (FUSE) is a software interface for developing file systems in user-mode without kernel interface programming. Crossmeta 45 | provides the same reference Linux FUSE API, so that programs can be ported with no or very little modifications. 46 | It comes with following classic "Hello, World" samples 47 | hello using high-level FUSE API 48 | hello_ll using low-level FUSE API 49 | sshfs is a popular FUSE file system client that allows secure seamless access to remote files with standard SSH account login, as if they were local files. 50 | 51 | #### Pseudo character devices 52 | Crossmeta provides character devices such as 53 | 54 |
55 |
zero
Can be used as source of endless zero 56 |
null
Can be used as output sink to discard data 57 |
loop
a pseudo device that makes ordinary file accessible as a block device 58 |
59 | The windows volume devices appear as BSD style disks 60 | da0, da0s1 .. da1, .. etc 61 |
62 | 63 | #### Using file images VHD 64 | Attach or Detach of VHD image in Windows Disk management will appear as PnP disk inside Crossmeta. 65 | Other ordinary flat file images can be used as disk with loop pseud-device support and losetup command. 66 | 67 | ## Crossmeta Kernel 68 | Standard NT kernel mode file system driver with BSD VFS operations. 69 | It includes 70 | - namecache for caching name lookups 71 | - buffercache for directory or metadata caching 72 | - regular files uses NT Cache manager (pagecache) 73 | 74 | #### rootfs 75 | The grandfather of UNIX file systems, FFS/UFS, serves as default root file system in Crossmeta. 76 | 77 | #### winfs 78 | The loop back filesystem in Crossmeta to access other Windows files systems NTFS, etc. Winfs automatically reflects any modification made outside Crossmeta to these native file systems. 79 | 80 | #### regfs 81 | Readonly access to NT Registry hive. With this file system the registry keys appear as directories and the registry values appear as files with binary data. 82 | 83 | #### devfs 84 | Pseudo file system to expose Crossmeta devices, /dev/null, zero, loop, da0.. etc 85 | 86 | #### dmesg 87 | Crossmeta provides the standard klog device for viewing the kernel informational or debug messages from Crossmeta. 88 | 89 | #### sysctl 90 | To retrieve kernel state and allows processes with appropriate privilege to set kernel state.The state to be retrieved or 91 | set is described using a ``Management Information Base'' (``MIB'') style 92 | name, described as a dotted set of components. 93 | 94 | #### User mapping 95 | Crossmeta uses one-to-one mapping of NT user name (SID) to BSD passwd db file withe same username, uid, gid, and groups. Unknown names are mapped to _nobody_ 96 | 97 | #### Network 98 | NFS file system 99 | Crossmeta provides version 2 and 3 NFS server and client services over UDP protocol. 100 | 101 | ## Summary 102 | 103 | Crossmeta provides practical POSIX environment in Windows Environment that is easy to install and use and is available in very small memory footprint. 104 | The common UNIX utilities are hard-linked files that is less than 300KB and the Crossmeta kernel driver is also under 300KB. 105 | -------------------------------------------------------------------------------- /TESTING.md: -------------------------------------------------------------------------------- 1 | ## Regression Testing 2 | Crossmeta uses Connectathon test suite to verify basic POSIX compliant file system operations. Though the tests were orignially provided to verify NFS operations, it can also be used for testing filesystems other than NFS. 3 | It also uses its own busybox ash shell for running the regression.sh shell script! 4 | 5 | 1. Install latest [Crossmeta Fuse](https://github.com/crossmeta/cxfuse/releases/latest) on *Windows Server 2016 build 14393* 6 | 2. Start Crossmeta drivers 7 | ```powershell 8 | c:\program files\crossmeta> service_crossmeta start 9 | ``` 10 | 3. Create temporary directory where file system image will setup in loop disk. 11 | ```powershell 12 | c:\ mkdir c:\temp 13 | ``` 14 | 4. Download cthon04 test suite ported to Crossmeta and save it to Crossmeta drive v:\tmp 15 | 5. From Crossmeta shell (busybox) extract files 16 | ```shell 17 | # cd /tmp 18 | # gzip -cd cthon04_tests.tgz | tar xf - 19 | # cd cthon4 20 | 6. Before running regression.sh shell script review to make sure the variable settings are okay. 21 | 7. Run the regression suite 22 | ``shell 23 | # cd /tmp/cth0n04 24 | # ./regression.sh 25 | ``` 26 | 8. After any failure rerun requires cleanup as follows 27 | ```console 28 | # umount /mnt 29 | # losetup -d loop1 30 | # cmd /c del c:\\temp\\cxvfstest.img 31 | ``` 32 | 9. Watch the progress of test ouptut 33 | ```console 34 | # ./regression.sh 35 | meta-data=/dev/loop1 isize=256 agcount=6, agsize=4096 blks 36 | = sectsz=512 attr=0 37 | data = bsize=4096 blocks=24576, imaxpct=25 38 | = sunit=0 swidth=0 blks, unwritten=1 39 | naming =version 2 bsize=4096 40 | log =internal log bsize=4096 blocks=1200, version=1 41 | = sectsz=512 sunit=0 blks 42 | realtime =none extsz=65536 blocks=0, rtextents=0 43 | Fri Mar 22 23:26:33 GST 2019 Start xfs file system testing ... 44 | Starting BASIC tests: test directory /mnt/testdir (arg: -t) 45 | 46 | ./test1: File and directory creation test 47 | created 155 files 62 directories 5 levels deep in 0.1 seconds 48 | ./test1 ok. 49 | 50 | ./test2: File and directory removal test 51 | removed 155 files 62 directories 5 levels deep in 0.1 seconds 52 | ./test2 ok. 53 | 54 | ./test3: lookups across mount point 55 | 500 getcwd and stat calls in 0.3 seconds 56 | ./test3 ok. 57 | 58 | ./test4: setattr, getattr, and lookup 59 | 1000 chmods and stats on 10 files in 0.1 seconds 60 | ./test4 ok. 61 | 62 | ./test5: read and write 63 | wrote 1048576 byte file 10 times in 0.1 seconds (699050666 bytes/sec) 64 | read 1048576 byte file 10 times in 0.0 seconds (> 10485760 bytes/sec) 65 | ./test5 ok. 66 | TESTARG=-t 67 | ./test6: readdir 68 | 20500 entries read, 200 files in 0.1 seconds 69 | ./test6 ok. 70 | 71 | ./test7: link and rename 72 | 200 renames and links on 10 files in 0.0 seconds 73 | ./test7 ok. 74 | 75 | ./test8: symlink and readlink 76 | 400 symlinks and readlinks on 10 files in 0.1 seconds 77 | ./test8 ok. 78 | 79 | ./test9: statfs 80 | 1500 statfs calls in 0.1 seconds 81 | ./test9 ok. 82 | 83 | Congratulations, you passed the basic tests! 84 | Fri Mar 22 23:26:33 GST 2019 Done xfs file system testing ... 85 | Fri Mar 22 23:26:33 GST 2019 Begin umount /mnt type xfs ... 86 | Fri Mar 22 23:26:34 GST 2019 Done umount /mnt type xfs ... 87 | /dev/loop1 is not a block special device. 88 | Proceed anyway? (y,n) Filesystem label= 89 | OS type: Linux 90 | Block size=1024 (log=0) 91 | Fragment size=1024 (log=0) 92 | 25688 inodes, 102400 blocks 93 | 5120 blocks (5.00%) reserved for the super user 94 | First data block=1 95 | 13 block groups 96 | 8192 blocks per group, 8192 fragments per group 97 | 1976 inodes per group 98 | Superblock backups stored on blocks: 99 | 8193, 24577, 40961, 57345, 73729 100 | 101 | Writing inode tables: done 102 | Writing superblocks and filesystem accounting information: done 103 | 104 | This filesystem will be automatically checked every 39 mounts or 105 | 180 days, whichever comes first. Use tune2fs -c or -i to override. 106 | Fri Mar 22 23:26:34 GST 2019 Start ext4 file system testing ... 107 | Starting BASIC tests: test directory /mnt/testdir (arg: -t) 108 | 109 | ./test1: File and directory creation test 110 | created 155 files 62 directories 5 levels deep in 0.1 seconds 111 | ./test1 ok. 112 | 113 | ./test2: File and directory removal test 114 | removed 155 files 62 directories 5 levels deep in 0.1 seconds 115 | ./test2 ok. 116 | 117 | ./test3: lookups across mount point 118 | 500 getcwd and stat calls in 0.1 seconds 119 | ./test3 ok. 120 | 121 | ./test4: setattr, getattr, and lookup 122 | 1000 chmods and stats on 10 files in 0.0 seconds 123 | ./test4 ok. 124 | 125 | ./test5: read and write 126 | wrote 1048576 byte file 10 times in 0.4 seconds (223101276 bytes/sec) 127 | read 1048576 byte file 10 times in 0.0 seconds (> 10485760 bytes/sec) 128 | ./test5 ok. 129 | TESTARG=-t 130 | ./test6: readdir 131 | 20500 entries read, 200 files in 0.0 seconds 132 | ./test6 ok. 133 | 134 | ./test7: link and rename 135 | 200 renames and links on 10 files in 0.0 seconds 136 | ./test7 ok. 137 | 138 | ./test8: symlink and readlink 139 | 400 symlinks and readlinks on 10 files in 0.0 seconds 140 | ./test8 ok. 141 | 142 | ./test9: statfs 143 | 1500 statfs calls in 0.0 seconds 144 | ./test9 ok. 145 | 146 | Congratulations, you passed the basic tests! 147 | Fri Mar 22 23:26:35 GST 2019 Done ext4 file system testing ... 148 | Fri Mar 22 23:26:35 GST 2019 Begin umount /mnt type ext4 ... 149 | umount: unmount of /mnt failed: Resource device 150 | Fri Mar 22 23:26:37 GST 2019 Done umount /mnt type ext4 ... 151 | /dev/loop1: 204800 sectors in 50 cylinders of 1 tracks, 4096 sectors 152 | 100.0MB in 4 cyl groups (16 c/g, 32.00MB/g, 6272 i/g) 153 | super-block backups (for fsck -b #) at: 154 | 32, 65568, 131104, 196640 155 | Fri Mar 22 23:26:37 GST 2019 Start ufs file system testing ... 156 | Starting BASIC tests: test directory /mnt/testdir (arg: -t) 157 | 158 | ./test1: File and directory creation test 159 | created 155 files 62 directories 5 levels deep in 0.1 seconds 160 | ./test1 ok. 161 | 162 | ./test2: File and directory removal test 163 | removed 155 files 62 directories 5 levels deep in 0.0 seconds 164 | ./test2 ok. 165 | 166 | ./test3: lookups across mount point 167 | 500 getcwd and stat calls in 0.1 seconds 168 | ./test3 ok. 169 | 170 | ./test4: setattr, getattr, and lookup 171 | 1000 chmods and stats on 10 files in 0.1 seconds 172 | ./test4 ok. 173 | 174 | ./test5: read and write 175 | wrote 1048576 byte file 10 times in 0.0 seconds (> 10485760 bytes/sec) 176 | read 1048576 byte file 10 times in 0.1 seconds (655360000 bytes/sec) 177 | ./test5 ok. 178 | TESTARG=-t 179 | ./test6: readdir 180 | 20500 entries read, 200 files in 0.0 seconds 181 | ./test6 ok. 182 | 183 | ./test7: link and rename 184 | 200 renames and links on 10 files in 0.0 seconds 185 | ./test7 ok. 186 | 187 | ./test8: symlink and readlink 188 | 400 symlinks and readlinks on 10 files in 0.0 seconds 189 | ./test8 ok. 190 | 191 | ./test9: statfs 192 | 1500 statfs calls in 0.0 seconds 193 | ./test9 ok. 194 | 195 | Congratulations, you passed the basic tests! 196 | Fri Mar 22 23:26:38 GST 2019 Done ufs file system testing ... 197 | Fri Mar 22 23:26:38 GST 2019 Begin umount /mnt type ufs ... 198 | Fri Mar 22 23:26:38 GST 2019 Done umount /mnt type ufs ... 199 | /dev/loop1 is not a block special device. 200 | Proceed anyway? (y,n) Filesystem label= 201 | OS type: Linux 202 | Block size=1024 (log=0) 203 | Fragment size=1024 (log=0) 204 | 25688 inodes, 102400 blocks 205 | 5120 blocks (5.00%) reserved for the super user 206 | First data block=1 207 | 13 block groups 208 | 8192 blocks per group, 8192 fragments per group 209 | 1976 inodes per group 210 | Superblock backups stored on blocks: 211 | 8193, 24577, 40961, 57345, 73729 212 | 213 | Writing inode tables: done 214 | Writing superblocks and filesystem accounting information: done 215 | 216 | This filesystem will be automatically checked every 23 mounts or 217 | 180 days, whichever comes first. Use tune2fs -c or -i to override. 218 | Fri Mar 22 23:26:38 GST 2019 Start ext2 file system testing ... 219 | Starting BASIC tests: test directory /mnt/testdir (arg: -t) 220 | 221 | ./test1: File and directory creation test 222 | created 155 files 62 directories 5 levels deep in 0.1 seconds 223 | ./test1 ok. 224 | 225 | ./test2: File and directory removal test 226 | removed 155 files 62 directories 5 levels deep in 0.0 seconds 227 | ./test2 ok. 228 | 229 | ./test3: lookups across mount point 230 | 500 getcwd and stat calls in 0.3 seconds 231 | ./test3 ok. 232 | 233 | ./test4: setattr, getattr, and lookup 234 | 1000 chmods and stats on 10 files in 0.0 seconds 235 | ./test4 ok. 236 | 237 | ./test5: read and write 238 | wrote 1048576 byte file 10 times in 0.1 seconds (655360000 bytes/sec) 239 | read 1048576 byte file 10 times in 0.0 seconds (> 10485760 bytes/sec) 240 | ./test5 ok. 241 | TESTARG=-t 242 | ./test6: readdir 243 | 20500 entries read, 200 files in 0.0 seconds 244 | ./test6 ok. 245 | 246 | ./test7: link and rename 247 | 200 renames and links on 10 files in 0.0 seconds 248 | ./test7 ok. 249 | 250 | ./test8: symlink and readlink 251 | 400 symlinks and readlinks on 10 files in 0.0 seconds 252 | ./test8 ok. 253 | 254 | ./test9: statfs 255 | 1500 statfs calls in 0.0 seconds 256 | ./test9 ok. 257 | 258 | Congratulations, you passed the basic tests! 259 | Fri Mar 22 23:26:39 GST 2019 Done ext2 file system testing ... 260 | Fri Mar 22 23:26:39 GST 2019 Begin umount /mnt type ext2 ... 261 | umount: unmount of /mnt failed: Resource device 262 | Fri Mar 22 23:26:41 GST 2019 Done umount /mnt type ext2 ... 263 | Removing image file c:\temp\cxvfstest.img 264 | # 265 | ``` 266 | 267 | 268 | -------------------------------------------------------------------------------- /regression.sh: -------------------------------------------------------------------------------- 1 | # 2 | # CROSSMETA basic regression test script 3 | # (c) Copyright Sam Sammandam suprasam _at_ crossmeta.org 4 | # 5 | 6 | FSTYPES="xfs ext4 ufs ext2" 7 | FSIMAGE='c:\temp\cxvfstest.img' 8 | SIZE_BYTES=$((100*1024*1024)) 9 | MNTPOINT=/mnt 10 | CTHON=/tmp/cthon04 11 | LOOPDEV=/dev/loop1 12 | LOSETUP=/bin/losetup 13 | 14 | if [ -e $LOOPDEV ] ; then 15 | echo "Sorry $LOOPDEV already taken!" 16 | exit 1 17 | fi 18 | set -e 19 | 20 | $LOSETUP -s $SIZE_BYTES -n $LOOPDEV $FSIMAGE 21 | for fs in $FSTYPES ; do 22 | case $fs in 23 | ufs|ffs) MKFS=/bin/newfs 24 | ;; 25 | 26 | xfs) MKFS=/bin/xfs_mkfs.exe 27 | ;; 28 | 29 | ext*) MKFS=/bin/mke2fs.exe 30 | ;; 31 | 32 | \?) 33 | echo "File system $fs not supported for testing." 34 | exit 1 35 | esac 36 | 37 | if [ "$fs" = "ext2" -o "$fs" = "ext4" ] ; then 38 | echo y | $MKFS $LOOPDEV |tee 39 | else 40 | $MKFS $LOOPDEV 41 | fi 42 | 43 | mount -t $fs $LOOPDEV $MNTPOINT 44 | 45 | echo "`date` Start $fs file system testing ..." 46 | cd $CTHON/basic 47 | NFSTESTDIR=$MNTPOINT/testdir ./runtests -t 48 | echo "`date` Done $fs file system testing ..." 49 | 50 | echo "`date` Begin umount $MNTPOINT type $fs ..." 51 | set +e 52 | retry=0 53 | while [ $retry -lt 10 ] ; do 54 | umount $MNTPOINT && break 55 | sleep 2 56 | retry=$(($retry+1)) 57 | done 58 | echo "`date` Done umount $MNTPOINT type $fs ..." 59 | set -e 60 | 61 | done 62 | set +e 63 | $LOSETUP -d $LOOPDEV 64 | /bin/dmesg.exe > dmesg-`date +%Y-%m-%d.%H:%M:%S` 65 | 66 | echo Removing image file $FSIMAGE 67 | cmd /c del $FSIMAGE 68 | 69 | 70 | --------------------------------------------------------------------------------