├── spec ├── spec_helper.cr └── flp_spec.cr ├── .gitignore ├── src ├── flp │ ├── version.cr │ ├── formatter │ │ ├── base.cr │ │ └── html.cr │ ├── errors.cr │ ├── policy │ │ ├── base.cr │ │ ├── header_length.cr │ │ ├── header_format.cr │ │ ├── header_identifier.cr │ │ └── data_identifier.cr │ ├── project.cr │ ├── application.cr │ └── parser.cr └── flp-viewer.cr ├── shard.lock ├── .travis.yml ├── .editorconfig ├── shard.yml ├── Makefile ├── README.md ├── LICENSE └── doc └── Reverse-Engineering.md /spec/spec_helper.cr: -------------------------------------------------------------------------------- 1 | require "spec" 2 | require "../src/flp" 3 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | lib/ 2 | bin/ 3 | tmp/ 4 | build/ 5 | .shards/ 6 | *.dwarf 7 | -------------------------------------------------------------------------------- /src/flp/version.cr: -------------------------------------------------------------------------------- 1 | module FLP 2 | VERSION = "0.0.1" 3 | end 4 | 5 | -------------------------------------------------------------------------------- /src/flp-viewer.cr: -------------------------------------------------------------------------------- 1 | require "./flp/application" 2 | 3 | FLP::Application.call(ARGV) 4 | 5 | -------------------------------------------------------------------------------- /src/flp/formatter/base.cr: -------------------------------------------------------------------------------- 1 | abstract class FLP::Formatter::Base 2 | 3 | abstract def to_s(io) 4 | 5 | end 6 | 7 | -------------------------------------------------------------------------------- /shard.lock: -------------------------------------------------------------------------------- 1 | version: 1.0 2 | shards: 3 | html_builder: 4 | github: crystal-lang/html_builder 5 | version: 0.2.2 6 | 7 | -------------------------------------------------------------------------------- /src/flp/errors.cr: -------------------------------------------------------------------------------- 1 | module FLP 2 | 3 | class Error < Exception; end 4 | 5 | class ParseError < Error; end 6 | 7 | end 8 | 9 | -------------------------------------------------------------------------------- /spec/flp_spec.cr: -------------------------------------------------------------------------------- 1 | require "./spec_helper" 2 | 3 | describe Flp do 4 | # TODO: Write tests 5 | 6 | it "works" do 7 | false.should eq(true) 8 | end 9 | end 10 | -------------------------------------------------------------------------------- /src/flp/policy/base.cr: -------------------------------------------------------------------------------- 1 | abstract class FLP::Policy::Base 2 | 3 | def self.call(value) 4 | new.call(value) 5 | end 6 | 7 | abstract def call(value) 8 | 9 | end 10 | 11 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: crystal 2 | 3 | # Uncomment the following if you'd like Travis to run specs and check code formatting 4 | # script: 5 | # - crystal spec 6 | # - crystal tool format --check 7 | -------------------------------------------------------------------------------- /.editorconfig: -------------------------------------------------------------------------------- 1 | root = true 2 | 3 | [*.cr] 4 | charset = utf-8 5 | end_of_line = lf 6 | insert_final_newline = true 7 | indent_style = space 8 | indent_size = 2 9 | trim_trailing_whitespace = true 10 | -------------------------------------------------------------------------------- /src/flp/project.cr: -------------------------------------------------------------------------------- 1 | class FLP::Project 2 | 3 | def initialize 4 | end 5 | 6 | property path : String = "" 7 | property channels : UInt16 = 0 8 | property ppq : UInt16 = 0 9 | property started_at : Time? 10 | property work_time : Time::Span? 11 | 12 | end 13 | 14 | -------------------------------------------------------------------------------- /shard.yml: -------------------------------------------------------------------------------- 1 | name: flp-viewer 2 | version: 0.0.1 3 | 4 | authors: 5 | - Ryan Scott Lewis 6 | 7 | targets: 8 | flp: 9 | main: src/flp.cr 10 | 11 | crystal: 0.30.1 12 | 13 | license: MIT 14 | 15 | dependencies: 16 | html_builder: 17 | github: crystal-lang/html_builder 18 | -------------------------------------------------------------------------------- /src/flp/policy/header_length.cr: -------------------------------------------------------------------------------- 1 | require "./base" 2 | 3 | class FLP::Policy::HeaderLength < FLP::Policy::Base 4 | 5 | VALID_HEADER_LENGTH = 6 6 | MESSAGE = "Invalid header length" 7 | 8 | def call(value) 9 | raise ParseError.new(MESSAGE) unless value == VALID_HEADER_LENGTH 10 | 11 | value 12 | end 13 | 14 | end 15 | 16 | -------------------------------------------------------------------------------- /src/flp/policy/header_format.cr: -------------------------------------------------------------------------------- 1 | require "./base" 2 | 3 | class FLP::Policy::HeaderFormat < FLP::Policy::Base 4 | 5 | VALID_HEADER_FORMAT = 0 # 0 for full song. It's unknown what the other possible values are 6 | MESSAGE = "Invalid header format" 7 | 8 | def call(value) 9 | raise ParseError.new(MESSAGE) unless value == VALID_HEADER_FORMAT 10 | 11 | value 12 | end 13 | 14 | end 15 | 16 | -------------------------------------------------------------------------------- /src/flp/policy/header_identifier.cr: -------------------------------------------------------------------------------- 1 | require "./base" 2 | 3 | class FLP::Policy::HeaderIdentifier < FLP::Policy::Base 4 | 5 | VALID_HEADER_IDENTIFIER = 1684556870 # "FLhd" as 32-bit little-endian integer 6 | MESSAGE = "Invalid header identifier" 7 | 8 | def call(value) 9 | raise ParseError.new(MESSAGE) unless value == VALID_HEADER_IDENTIFIER 10 | 11 | value 12 | end 13 | 14 | end 15 | 16 | -------------------------------------------------------------------------------- /src/flp/policy/data_identifier.cr: -------------------------------------------------------------------------------- 1 | require "./base" 2 | 3 | # TODO UNUSED 4 | class FLP::Policy::DataIdentifier < FLP::Policy::Base 5 | 6 | VALID_DATA_IDENTIFIER = 1952730182 # "FLdt" as 32-bit little-endian integer 7 | MESSAGE = "Invalid data identifier" 8 | 9 | def call(value) 10 | raise ParseError.new(MESSAGE) unless value == VALID_DATA_IDENTIFIER 11 | 12 | value 13 | end 14 | 15 | end 16 | 17 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | SRCDIR = src 2 | BUILDDIR = build 3 | 4 | APP_NAME = $(shell basename $(dir $(realpath $(firstword $(MAKEFILE_LIST))))) 5 | APP_CR = $(shell find $(SRCDIR) -name "*.cr") 6 | APP_MAIN = $(SRCDIR)/$(APP_NAME).cr 7 | APP_EXE = $(BUILDDIR)/$(APP_NAME) 8 | 9 | .PHONY: all build 10 | 11 | all: build 12 | 13 | build: $(APP_EXE) 14 | 15 | $(APP_EXE): $(APP_CR) | $(BUILDDIR)/ 16 | crystal build -o $@ $(APP_MAIN) 17 | 18 | $(BUILDDIR)/: 19 | mkdir -p $@ 20 | 21 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # FLP Viewer 2 | 3 | FL Studio FLP file format parser and viewer. 4 | 5 | ## Installation 6 | 7 | TODO: Write installation instructions here 8 | 9 | ## Usage 10 | 11 | At the moment, there is only HTML output: 12 | 13 | ```sh 14 | $ flp-viewer ~/projects/*.flp > output.html 15 | ``` 16 | 17 | ## Development 18 | 19 | TODO: Write development instructions here 20 | 21 | ## Contributing 22 | 23 | 1. Fork it () 24 | 2. Create your feature branch (`git checkout -b my-new-feature`) 25 | 3. Commit your changes (`git commit -am 'Add some feature'`) 26 | 4. Push to the branch (`git push origin my-new-feature`) 27 | 5. Create a new Pull Request 28 | 29 | ## Contributors 30 | 31 | * [Ryan Scott Lewis](https://github.com/RyanScottLewis) - creator and maintainer 32 | 33 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2019 Ryan Scott Lewis 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in 13 | all copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 21 | THE SOFTWARE. 22 | -------------------------------------------------------------------------------- /src/flp/application.cr: -------------------------------------------------------------------------------- 1 | require "./parser" 2 | require "./errors" 3 | require "./formatter/html" 4 | 5 | module FLP 6 | class Application 7 | 8 | def self.call(arguments) 9 | new(arguments).call 10 | end 11 | 12 | @arguments : Array(String) 13 | 14 | def initialize(@arguments) 15 | end 16 | 17 | def call 18 | started_at = Time.now 19 | 20 | validate_arguments_length 21 | 22 | projects = @arguments.map do |path| 23 | validate_path_exists(path) 24 | parse_project(path) 25 | end 26 | 27 | build_duration = Time.now - started_at 28 | 29 | html = generate_html(projects, build_duration) 30 | 31 | puts html 32 | end 33 | 34 | protected def print_usage 35 | puts "Usage: flp PATHS..." 36 | end 37 | 38 | protected def validate_arguments_length 39 | return unless @arguments.empty? 40 | 41 | print_usage 42 | exit 1 43 | end 44 | 45 | protected def validate_path_exists(path) 46 | return if File.exists?(path) 47 | 48 | puts "Error: Path '#{path}' does not exist" 49 | exit 1 50 | end 51 | 52 | protected def parse_project(path) 53 | project = Parser.parse(path) 54 | 55 | project.path = path 56 | 57 | project 58 | rescue error : Error 59 | puts "Error: #{error}" 60 | exit 1 61 | end 62 | 63 | protected def generate_html(projects, build_duration) 64 | Formatter::HTML.new(projects, build_duration).to_s 65 | end 66 | 67 | end 68 | end 69 | -------------------------------------------------------------------------------- /src/flp/formatter/html.cr: -------------------------------------------------------------------------------- 1 | require "../version" 2 | require "../project" 3 | require "./base" 4 | 5 | require "html_builder" 6 | 7 | class FLP::Formatter::HTML < FLP::Formatter::Base 8 | 9 | BOOTSTRAP_VERSION = "4.3.1" 10 | JQUERY_VERSION = "3.3.1" 11 | DATATABLES_VERSION = "1.10.19" 12 | 13 | @projects : Array(Project) 14 | @build_duration : Time::Span 15 | 16 | def initialize(@projects, @build_duration) 17 | end 18 | 19 | # TODO: Holy fuck lol just use ECR 20 | def to_s(io) 21 | io << ::HTML.build do |builder| 22 | 23 | doctype 24 | 25 | html do 26 | 27 | head do 28 | 29 | title { text "FL Studio Projects" } 30 | link(rel: "stylesheet", href: "https://stackpath.bootstrapcdn.com/bootstrap/#{BOOTSTRAP_VERSION}/css/bootstrap.min.css", integrity: "sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T", crossorigin: "anonymous") 31 | link(rel: "stylesheet", href: "https://cdn.datatables.net/#{DATATABLES_VERSION}/css/dataTables.bootstrap4.min.css", crossorigin: "anonymous") 32 | 33 | script(src: "https://code.jquery.com/jquery-#{JQUERY_VERSION}.slim.min.js", integrity: "sha384-q8i/X+965DzO0rT7abK41JStQIAqVgRVzpbzo5smXKp4YfRvH+8abtTE1Pi6jizo", crossorigin: "anonymous") {} 34 | script(src: "https://cdn.datatables.net/#{DATATABLES_VERSION}/js/jquery.dataTables.min.js", crossorigin: "anonymous") {} 35 | script(src: "https://cdn.datatables.net/#{DATATABLES_VERSION}/js/dataTables.bootstrap4.min.js", crossorigin: "anonymous") {} 36 | 37 | script do 38 | # This should be named `raw` or something, because it just appends the string to the output. 39 | html "$(document).ready( function () { $('main table').DataTable(); } );" 40 | end 41 | 42 | end 43 | 44 | body(class: "container-fluid py-3") do 45 | 46 | header(class: "d-flex justify-content-between") do 47 | h1 { text "FL Studio Projects" } 48 | end 49 | 50 | hr 51 | 52 | main do 53 | 54 | table(class: "table table-sm table-bordered table-hover table-responsive") do 55 | thead(class: "thead-dark") do 56 | tr do 57 | th(scope: "col") { text "Path" } 58 | th(scope: "col") { text "Channels" } 59 | th(scope: "col") { text "Started At" } 60 | th(scope: "col") { text "Work Time" } 61 | end 62 | end 63 | 64 | tbody do 65 | @projects.each do |project| 66 | 67 | tr do 68 | td { text project.path } 69 | td { text project.channels.inspect } 70 | td(class: "text-nowrap") { text project.started_at.inspect } 71 | td(class: "text-nowrap") { text project.work_time.inspect } 72 | end 73 | 74 | end 75 | end 76 | end # table 77 | 78 | end # main 79 | 80 | hr 81 | 82 | footer do 83 | div(class: "text-muted d-flex justify-content-between") do 84 | 85 | div do 86 | text "Generated in #{@build_duration.total_seconds}s" 87 | end 88 | 89 | div do 90 | div { text "Powered by:" } 91 | 92 | div(class: "ml-3") do 93 | 94 | powered_by = [ # TODO: Move to CONST 95 | { url: "https://github.com/RyanScottLewis/flp-viewer", name: "FLP Viewer", version: VERSION }, 96 | { url: "https://crystal-lang.org", name: "Crystal", version: Crystal::VERSION }, 97 | { url: "https://getbootstrap.com", name: "Bootstrap", version: BOOTSTRAP_VERSION }, 98 | { url: "https://jquery.com", name: "jQuery", version: JQUERY_VERSION }, 99 | { url: "https://datatables.net", name: "DataTables", version: DATATABLES_VERSION }, 100 | ] 101 | 102 | powered_by.each do |item| 103 | 104 | div(class: "row") do 105 | div(class: "col-6 text-nowrap") do 106 | a(href: item[:url], target: "_blank") { text item[:name] } 107 | end 108 | 109 | div(class: "col-6 text-nowrap text-right") do 110 | span { text item[:version] } 111 | end 112 | end 113 | 114 | end 115 | 116 | end 117 | 118 | end 119 | 120 | end 121 | end # footer 122 | 123 | end 124 | 125 | end 126 | 127 | end 128 | end 129 | 130 | end 131 | -------------------------------------------------------------------------------- /src/flp/parser.cr: -------------------------------------------------------------------------------- 1 | require "./errors" 2 | require "./policy/*" 3 | require "./project" 4 | 5 | class FLP::Parser 6 | 7 | enum Event 8 | Byte = 0 9 | Enabled = 0 10 | NoteOn = 1 11 | Vol = 2 12 | Pan = 3 13 | MIDIChan = 4 14 | MIDINote = 5 15 | MIDIPatch = 6 16 | MIDIBank = 7 17 | LoopActive = 9 18 | ShowInfo = 10 19 | Shuffle = 11 20 | MainVol = 12 21 | Stretch = 13 22 | Pitchable = 14 23 | Zipped = 15 24 | Delay_Flags = 16 25 | PatLength = 17 26 | BlockLength = 18 27 | UseLoopPoints = 19 28 | LoopType = 20 29 | ChanType = 21 30 | MixSliceNum = 22 31 | EffectChannelMuted = 27 32 | 33 | Word = 64 34 | NewChan = Word 35 | NewPat = Word + 1 36 | Tempo = Word + 2 37 | CurrentPatNum = Word + 3 38 | PatData = Word + 4 39 | FX = Word + 5 40 | Fade_Stereo = Word + 6 41 | CutOff = Word + 7 42 | DotVol = Word + 8 43 | DotPan = Word + 9 44 | PreAmp = Word + 10 45 | Decay = Word + 11 46 | Attack = Word + 12 47 | DotNote = Word + 13 48 | DotPitch = Word + 14 49 | DotMix = Word + 15 50 | MainPitch = Word + 16 51 | RandChan = Word + 17 52 | MixChan = Word + 18 53 | Resonance = Word + 19 54 | LoopBar = Word + 20 55 | StDel = Word + 21 56 | FX3 = Word + 22 57 | DotReso = Word + 23 58 | DotCutOff = Word + 24 59 | ShiftDelay = Word + 25 60 | LoopEndBar = Word + 26 61 | Dot = Word + 27 62 | DotShift = Word + 28 63 | LayerChans = Word + 30 64 | 65 | Int = 128 66 | Color = Int 67 | PlayListItem = Int + 1 68 | Echo = Int + 2 69 | FXSine = Int + 3 70 | CutCutBy = Int + 4 71 | WindowH = Int + 5 72 | MiddleNote = Int + 7 73 | Reserved = Int + 8 74 | MainResoCutOff = Int + 9 75 | DelayReso = Int + 10 76 | Reverb = Int + 11 77 | IntStretch = Int + 12 78 | SSNote = Int + 13 79 | FineTune = Int + 14 80 | 81 | Undef = 192 82 | Text = Undef 83 | Text_ChanName = Text 84 | Text_PatName = Text + 1 85 | Text_Title = Text + 2 86 | Text_Comment = Text + 3 87 | Text_SampleFileName = Text + 4 88 | Text_URL = Text + 5 89 | Text_CommentRTF = Text + 6 90 | Text_Version = Text + 7 91 | Text_PluginName = Text + 9 92 | Text_EffectChanName = Text + 12 93 | Text_MIDICtrls = Text + 16 94 | Text_Delay = Text + 17 95 | Text_TS404Params = Text + 18 96 | Text_DelayLine = Text + 19 97 | Text_NewPlugin = Text + 20 98 | Text_PluginParams = Text + 21 99 | Text_ChanParams = Text + 23 100 | Text_EnvLfoParams = Text + 26 101 | Text_BasicChanParams = Text + 27 102 | Text_OldFilterParams = Text + 28 103 | Text_AutomationData = Text + 31 104 | Text_PatternNotes = Text + 32 105 | Text_ChanGroupName = Text + 39 106 | Text_PlayListItems = Text + 41 107 | Text_Time = Text + 45 108 | end 109 | 110 | TIME_ORIGIN = Time.new(1899, 12, 30) 111 | 112 | def self.parse(path) 113 | new.parse(path) 114 | end 115 | 116 | def parse(path : String) 117 | File.open(path) { |io| parse(io) } 118 | end 119 | 120 | def parse(io : IO) 121 | parse_header(io) 122 | parse_project(io) 123 | end 124 | 125 | protected def read_uint16(io) 126 | io.read_bytes(UInt16, IO::ByteFormat::LittleEndian) 127 | end 128 | 129 | protected def read_uint32(io) 130 | io.read_bytes(UInt32, IO::ByteFormat::LittleEndian) 131 | end 132 | 133 | protected def parse_header(io) 134 | Policy::HeaderIdentifier.call(read_uint32(io)) 135 | Policy::HeaderLength.call(read_uint32(io)) 136 | Policy::HeaderFormat.call(read_uint16(io)) 137 | end 138 | 139 | protected def parse_project(io) 140 | project = Project.new 141 | 142 | project.channels = read_uint16(io) 143 | project.ppq = read_uint16(io) 144 | 145 | data_identifier = io.read_string(4) 146 | return project unless data_identifier == "FLdt" 147 | 148 | while event = parse_event(io) 149 | type, data = event 150 | 151 | if type == Event::Text_Time 152 | bytes = data.as(String).to_slice 153 | 154 | return project if bytes.size != 16 155 | 156 | started_at = IO::ByteFormat::LittleEndian.decode(Float64, bytes[0, 8]) 157 | started_at = Time::Span.new(1, 0, 0, 0) * started_at 158 | started_at = TIME_ORIGIN + started_at 159 | 160 | work_time = IO::ByteFormat::LittleEndian.decode(Float64, bytes[8, 8]) 161 | work_time = Time::Span.new(1, 0, 0, 0) * work_time 162 | 163 | project.started_at = started_at 164 | project.work_time = work_time 165 | 166 | break 167 | end 168 | 169 | end 170 | 171 | project 172 | end 173 | 174 | protected def parse_event(io) 175 | type = io.read_byte 176 | return if type.nil? 177 | type = Event.new(type.to_i32) 178 | 179 | data = io.read_byte 180 | return if data.nil? 181 | data = data.to_u32 182 | 183 | if (type >= Event::Word && type < Event::Text) 184 | data_partial = io.read_byte 185 | return if data_partial.nil? 186 | data_partial = data_partial.to_u32 187 | 188 | data = data | (data_partial << 8) 189 | end 190 | 191 | if (type >= Event::Int && type < Event::Text) 192 | data_partial = io.read_byte 193 | return if data_partial.nil? 194 | data_partial = data_partial.to_u32 195 | 196 | data = data | (data_partial << 16) 197 | 198 | data_partial = io.read_byte 199 | return if data_partial.nil? 200 | data_partial = data_partial.to_u32 201 | 202 | data = data | (data_partial << 24) 203 | end 204 | 205 | if (type >= Event::Text) 206 | length = data & 0x7F 207 | shift = 0 208 | 209 | while (data & 0x80) == 1 210 | data = io.read_byte 211 | return if data.nil? 212 | data = data.to_u32 213 | 214 | length = length | ((data & 0x7F) << (shift+=7)) 215 | end 216 | 217 | data = io.read_string(length) 218 | return if data.nil? 219 | end 220 | 221 | { type, data } 222 | end 223 | 224 | end 225 | 226 | -------------------------------------------------------------------------------- /doc/Reverse-Engineering.md: -------------------------------------------------------------------------------- 1 | # Reverse Engineering the FL Studio Proprietary File Format 2 | 3 | ## Goal 4 | 5 | I (along with many music producers) have a great deal of project files, some of which are very old, 6 | and I would like to weed out the crappy ones containing maybe just a beat or an experiment and only 7 | search for projects containing substance. 8 | 9 | FL Studio keeps track of project creation time and project working time, which are excellent metrics 10 | for finding such projects. 11 | 12 | Unfortunately, the proprietary FLP format is not exactly friendly. The little amount of documentation 13 | on it and the few open-source FLP parser implementations leave much to be desired. In fact, I can't 14 | find a single one which can actually parse project creation/working times. 15 | 16 | This information is obviously in the file, though. So, that means it must be reverse engineered. 17 | 18 | > Editors note: This changes during the course of writing this article, as FLPEdit can in fact parse 19 | > project times and is used as reference once found. 20 | 21 | ## Steps 22 | 23 | ### Create a Parser 24 | 25 | Luckily, there are implementations of FLP file format parsers out there. I decided on using Crystal 26 | as it's expressive but strongly typed, which is perfect for parsing binary data. Doing so in Ruby is 27 | a String manipulation nightmare waiting to happen. 28 | 29 | So, really all I did was port the parser to Crystal from the multiple different projects: 30 | 31 | * https://github.com/andrewrk/PyDaw (Python implementation) 32 | * https://github.com/monadgroup/FLParser (C-Sharp implementation) 33 | * https://github.com/LMMS/lmms (C++ implementation) 34 | * Sidenote: I had no idea LMMS could import FLP projects until now. 35 | * https://github.com/RoadCrewWorker/FLPEdit (C-Sharp implementation) 36 | * By far the best resource so far, found while creating this article. 37 | 38 | > Note how similar their sources are, you can tell they've all either derived from the same scarce 39 | > documentation, or from each other (which is our case). 40 | 41 | The parser works by identifying and reading the two chunks in order, then proceding to identify and 42 | read events based on their type until it reaches the end of the file. 43 | 44 | #### File Format 45 | 46 | The file format is a mess, but it breaks down like so: 47 | 48 | There are two chunks, the header chunk and the data chunk. 49 | 50 | The header chunk contains some basic information but was largely abandoned and can be ignored for 51 | the most part, only pertinent information contained in the header is the project channel count and 52 | PPQ (parts per quarter (?)) attribute. 53 | 54 | The data chunk simply contains a series of "events", identified by the first byte which determines 55 | how much data to read and seems to support 8, 16, and 32 bit integers and arbitrary text data. 56 | 57 | Most of these events have been identified, however there is none for what we actually want; which is 58 | the project creation and working times. 59 | 60 | ## Comparing Data 61 | 62 | So, to figure out these creation & working time values, I created two projects from the same 63 | template and saved them a few minutes apart from each other with no modifications. 64 | This way, the only difference in the two projects should be the project creation times. 65 | 66 | I converted these projects to hex dumps using `xxd a.flp > a.hex` and diff'd them with 67 | `nvim -d a.hex b.hex`. At this point, it was obvious that only a very small chunk of data has 68 | changed. 69 | 70 | I ran both projects through the parser and sure enough, only one event was different with the event 71 | identifier `237`: 72 | 73 | `a.flp` 74 | 75 | ``` 76 | [237, "J\f\u0002\u0013\xD9Y\xE5@\u0000\u0000\u0000\xA8\xACl:?"] 77 | ``` 78 | 79 | `b.flp` 80 | 81 | ``` 82 | [237, "\x80\xA1\xE5\u001C\xD9Y\xE5@\u0000\u0000\u0000@OI\u0012?"] 83 | ``` 84 | 85 | The data for this type was arbitrary text data, 16 bytes long. To identify what this chunk was, I 86 | opened op the first of my projects, kept active by selecting windows and such for a few minutes, 87 | then saved with a `-worked-on` postfix in the filename. I then diffed the parser output from those 88 | files. 89 | 90 | This showed that again that `237` event data was the only difference. This time, however, only the 91 | first 8 bytes remained the same: 92 | 93 | `a.flp` 94 | 95 | ``` 96 | [237, "J\f\u0002\u0013\xD9Y\xE5@\u0000\u0000\u0000\xA8\xACl:?"] 97 | ``` 98 | 99 | `a-worked-on.flp` 100 | 101 | ``` 102 | [237, "J\f\u0002\u0013\xD9Y\xE5@\u0000\u0000\u0000\u00184BL?"] 103 | ``` 104 | 105 | ## Identifying Data 106 | 107 | Here, I've printed out the data for the `237` event from many projects and formatted it in order to 108 | compare the bytes: 109 | 110 | ``` 111 | Offsets 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 16 112 | 113 | old-0.flp Bytes[234, 220, 179, 209, 215, 40, 229, 64, 0, 0, 64, 66, 151, 53, 139, 63] 114 | old-1.flp Bytes[166, 217, 159, 13, 88, 7, 229, 64, 0, 0, 128, 110, 3, 179, 171, 63] 115 | old-2.flp Bytes[206, 226, 0, 84, 1, 25, 229, 64, 0, 0, 208, 129, 37, 186, 164, 63] 116 | old-3.flp Bytes[106, 179, 117, 145, 156, 247, 228, 64, 0, 0, 208, 204, 19, 206, 175, 63] 117 | old-4.flp Bytes[230, 170, 239, 48, 141, 58, 229, 64, 0, 0, 96, 205, 213, 11, 180, 63] 118 | old-5.flp Bytes[148, 213, 17, 30, 31, 37, 229, 64, 0, 0, 160, 140, 166, 101, 154, 63] 119 | data/a.flp Bytes[74, 12, 2, 19, 217, 89, 229, 64, 0, 0, 0, 168, 172, 108, 58, 63] 120 | data/a-worked.flp Bytes[74, 12, 2, 19, 217, 89, 229, 64, 0, 0, 0, 24, 52, 66, 76, 63] 121 | data/b.flp Bytes[128, 161, 229, 28, 217, 89, 229, 64, 0, 0, 0, 64, 79, 73, 18, 63] 122 | ``` 123 | 124 | Things to note are: 125 | 126 | * `old-*` are random, old projects. 127 | * `a.flp` and `a-worked.flp` have the same creation time. 128 | * `a.flp`, `a-worked.flp`, and `b.flp` were created on the same date, minutes from each other. 129 | * Bytes at offsets 16 are identical for all projects. 130 | * Bytes at offsets 7-9 are identical for all projects. 131 | * Bytes at offset 6 is identical for all projects, with the exception of `old-3.flp`. 132 | 133 | ### Breakthrough 134 | 135 | It was around this time that I found the FLPEdit project by RoadCrewWorker on GitHub which actually 136 | seems to parse project times: 137 | 138 | ```cs 139 | // ... 140 | ID_Project_Time = FLP_Text + 45, 141 | // ... 142 | ``` 143 | 144 | The `FLP_Text` ID is `192`, which places `ID_Project_Time` at `192 + 45` which is `237`, our 145 | unidentified event ID! This confirms that it is actually the project time that we were inspecting. 146 | 147 | Now to identify the data using FLPEdit's source. 148 | 149 | Diving deeper into the FLPEdit source, I find the `FLPE_Project_Time` class, which is stating that 150 | the data is actually a Delphi time format. This seems to have an origin at the year `1900` and is 151 | stored in a `double`. The class even shows that the data is formatted in a 152 | 153 | > I would have never guessed this, having completely forgotten that FL Studio was programmed in Delphi. 154 | 155 | ## Parsing the Data 156 | 157 | Now that we have identified the data contained within our now identified project time data, we can 158 | move on to parsing this data. Crystal has alot of support for coding/decoding binary data, in this 159 | case a `double` which is a 64-bit floating point number (`Float64` in Crystal): 160 | 161 | ```cr 162 | TIME_ORIGIN = Time.new(1899, 12, 30) 163 | 164 | # ... 165 | 166 | bytes = data.as(String).to_slice 167 | 168 | start_date = IO::ByteFormat::LittleEndian.decode(Float64, bytes[0, 8]) 169 | start_date = Time::Span.new(1, 0, 0, 0) * start_date 170 | start_date = TIME_ORIGIN + start_date 171 | 172 | work_time = IO::ByteFormat::LittleEndian.decode(Float64, bytes[8, 8]) 173 | work_time = Time::Span.new(1, 0, 0, 0) * work_time 174 | ``` 175 | 176 | We're decoding the data from little endian as floats. The data is stored as `days` so we create a 177 | time span with the length of 1 day, and scale it by the value. 178 | 179 | In the case of start date, we add that time span to the Delphi origin date to retrieve the project 180 | creation time. 181 | 182 | ## Summary 183 | 184 | Well, this one was a bit of a struggle and truth be told, I was stuck trying to identify the contents 185 | of the then mysterious event `237`. Even to the point where I was scanning for data type just to try 186 | to see if any values could be recognized as a date time value or value partial. I'm truely indebted 187 | to RoadCrewWorker for somehow figuring out this Delphi date time debacle. 188 | 189 | However, I can at least take pride in correctly identifying the event type which needed to be parsed 190 | by analyzing differences in a binary file which has become the first tool I reach for in my reverse 191 | engineering toolbox. 192 | 193 | --------------------------------------------------------------------------------