Introduction
Logs are really helpful for troubleshooting, but they need to be easy to read. If you are working on a time-sensitive system and can’t waste time formatting your logs you need a parser/viewer for those logs.
If you are trying to spot something on the live tail from a log you need a way to figure out if the pattern you are looking for is there or not really fast.
Colors can help A LOT for eyeballing through your logs and getting actual information from them.
FIX protocol
Financial Information eXchange - FIX protocol is an electronic communications protocol initiated in 1992 for international real-time exchange of information related to securities transactions and markets and is still widely used those days.
Up to FIX 4.4, the messages are essentialy a tag-value collection of fields delimited by a control character.
Issues viewing message logs on the terminal
- The control char (0x01) does not have a drawable representation. So all fields are shown concatenated, causing some confusion when checking the messages;
- Since both tags and values have a bunch of numbers it is quite easy to mix them up;
Solution
This can be easily fixed with tr, sed and tput *nix tools.
1 | tail FIX.4.4-jrgcombr.messages.current.log | tr '\1' '|' | sed -E 's/\|([^=\|]+)=([^=\|]+)/\|'$(tput setaf 1)'\1'$(tput setaf 7)'='$(tput setaf 2)'\2'$(tput sgr0)'/g' |
Ok, not a pretty nor a handy command. Just put it an alias in your .bash_rc with a cool name.
Update 1
Found that less -R does not like tput sgr0 control chars, lets reset terminal just on the end of the line.
1 | alias fixcolor="tr '\1' '|' | sed -E 's/\|([^=\|]+)=([^=\|]+)/\|'$(tput setaf 1)'\1'$(tput setaf 7)'='$(tput setaf 2)'\2/g' | sed -E 's/\$/'$(tput sgr0)'/'" |
Then just use your new created alias.
1 | tail FIX.4.4-jrgcombr.messages.current.log | fixcolor |
Before
After
Extra tip
If you are looking for a specific pattern and do not want to skip other messages, just highlight this pattern.
1 | alias hl='grep -e "^"' |
1 | tail FIX.4.4-jrgcombr.messages.current.log | fixcolor | hl -e "venue" |
If it is not enough
If you feel like looking in more detail to some message you might want to try an online FIX parser
Be careful to not expose sensitive data!
Conclusion
This is low-hanging fruit. In the future, I might write a tool to convert tags to corresponding FIX dictionary names or event a JSON convert so we can hook up jq and apply filters.