The In Reference To header is to separate your lined items by project/case. To take off the header is to organize your lined items just by labor and expenses overall, and not organized further by project. If you still want to proceed, do the following:
Go to your invoice template and uncheck Group by Project.
Go to the Invoice Details for this particular invoice and uncheck Group Details by Project.