One day I was struggling to concatenate files generated by 3rd party utility for Essbase upload.
Uploading did broke on every single run and I cound not find the culprit until I opened the outline file with HxD HEX editor and found that extra bytes were added between each concatenated file.
To my suprise the 3rd party utility unloaded data from database in Unicode with BOM starting each separate file.
So before concatenating those files I had to remove BOM from them.
Thanks to this SO answer it was easily achievable with this python script.
# -*- coding: utf-8 -*- import os, sys, codecs BUFSIZE = 4096 BOMLEN = len(codecs.BOM_UTF8) path = sys.argv with open(path, "r+b") as fp: chunk = fp.read(BUFSIZE) if chunk.startswith(codecs.BOM_UTF8): i = 0 chunk = chunk[BOMLEN:] while chunk: fp.seek(i) fp.write(chunk) i += len(chunk) fp.seek(BOMLEN, os.SEEK_CUR) chunk = fp.read(BUFSIZE) fp.seek(-BOMLEN, os.SEEK_CUR) fp.truncate()