TODO: The Token field should be broken out into a separate char array to improve cache locality, since it s needed for costing when the rest of the fields are not.