Key principles and practices

Motivation

I know that reading time is a precious commodity. You should probably skim this page to see whether my principles and practices resonate with you.

If you do like them - keep reading. Even if you would not end up using my setup, maybe you will find some useful ideas to adopt.

NB: If you are just starting with hledger, this section could overwhelm you. If it happens -- skip it entirely for now. Everything contained here will be introduced in subsequent sections, and you can revisit this page at a later time.

However, if you are experienced (and potentially impatient) hledger user who does not want to read through the whole wiki or just wants a quick summary, then this section might be all you need.

Why Track Your Finances?

Keeping track of your finances empowers you to make informed decisions about your financial future. Just as logging your runs in a fitness app helps you improve your performance, or writing down debug logs from your program ensures that programming errors could be fixed quicker, detailed financial records provide critical insights when things don't go as planned.

In everyday life, you might not notice the benefits of maintaining financial records — until an unexpected event occurs. If a parcel goes missing, having a tracking record is invaluable. Similarly, if you miss your training targets or encounter a financial setback, your records can guide you through troubleshooting the issue.

Accurate financial records are crucial during times of hardship. They enable you to create a realistic budget and avoid overspending. At the same time, when your income is growing, detailed records allow you to invest wisely and strategically plan for your future. No matter your current financial situation, change is inevitable. With a solid record of your finances, you'll be well-prepared for those moments of transition, enabling deeper introspection and more effective planning.

Most importantly, maintaining comprehensive financial records is not just about accounting for every penny — it’s about securing a more predictable and controlled financial future.

Why Use Plain-Text Accounting

Plain-text accounting offers simplicity, stability, and flexibility.

Simplicity and Stability By storing your records in a simple, human-readable format, plain-text accounting makes your data both easy to access and easy to maintain. You don’t need specialized software to read or edit your .journal files, ensuring that your records remain usable today, tomorrow, and well into the future. In contrast, vendor-specific binary formats can become obsolete if the supporting software is discontinued, and cloud-based services might vanish at any time, potentially leaving you with inaccessible data.

Flexibility Plain-text files can be manipulated with any text-processing tool, such as grep, and can be easily converted into other formats like CSV or JSON. Tools like hledger, ledger, or beancount enable seamless exporting of your .journal files into these formats, allowing you to integrate your financial data with a variety of other applications. Moreover, the openness of plain text means you can write custom scripts or small programs, both for producing and consuming your accounting data.

What are the key practices adopted by this setup?

Version Control Everything and Maintain a Full "Chain of Custody"

Often, you’ll start with a file of financial transactions from your bank or brokerage and convert it into a .journal file for your plain-text accounting records. Once you’ve created your journal, the next crucial step is to put it (and all the files used to produce it) under version control. This practice:

• Protects you from accidental mistakes: You can experiment -- renaming accounts or making bulk edits -- without fear of permanently losing data.

• Preserves the evolution of your data: Retain all source files, intermediate conversion files, and detailed documentation of your conversion process. Whenever possible, automate your conversions and store the scripts or programs that perform these tasks. Similarly, when generating regular reports (balance sheets, income statements, etc.), create scripts to generate them and add these scripts to your version control system.

By automating every step and version-controlling your original files, intermediate artifacts, journals, and reports, you create a verifiable “chain of custody.” This chain ensures that every transformation of your data -- from the initial files provided by financial institutions to the final reports -- is reproducible and auditable/observable.

Refactor and Change Aggressively

With your entire "chain of custody" automated and under version control, you gain the freedom to modify any part of your system without fear of irreversibly breaking your records. For example:

• Adapting to new data formats: If your bank updates its statement format and offers more detailed information, you can update your conversion scripts to incorporate these changes. Then, regenerate historical data to compare reports, ensuring that key financial figures remain consistent.

• Improving data classification: Since you can re-generate journal files from your electronic statements, you can tweak import rules to improve transaction classification. This flexibility lets you refine your data organization without resorting to risky mass search-and-replace operations on finalized reports.

• Reorganizing your journals: Feel free to restructure your journal files as your needs evolve. With a fully version-controlled system, you can make substantial changes and review the updated reports to confirm that the modifications accurately reflect your financial history.

By embracing this iterative approach -- where any segment of your data chain can be updated and revalidated -- you build a system that not only evolves with your needs but also provides continuous confidence in the integrity and accuracy of your financial records.

Generate a Lot of Reports Automatically After Every Significant Change

After reconciling your accounts -- or even a subset of them -- automatically generate a bunch of reports. Create a one-click script that produces these reports and keep them under version control. This approach lets you quickly spot any irregularities or unexpected changes in your journals following updates.

Minimize Manual Entries

Manual data entry can be both time-consuming and error-prone, often taking more effort than managing your overall financial system. Consider whether every transaction needs to be tracked manually. For example, in my setup, all ATM transactions are recorded under the expenses:misc:cash account. Unless the total amount becomes significant, I don’t invest time in detailing each one. I strongly recommend a similar approach -- especially if you don't handle large amounts of cash -- so you can focus on the transactions that truly matter.

Split Your Journals by Year

Dividing your journals by year not only speeds up report generation but also allows you to “reset” your expenses at logical intervals. Both hledger and ledger offer a helpful equity (or close) command to simplify this process. With a bit of additional scripting, you can streamline the annual closing process even further, making your financial management more efficient.

Add More Details Over Time, as and When Needed

It’s easy to achieve a setup that requires a minimal effort to keep going: drop CSV statements into a directory, run a couple of scripts, review changes via version control, and then wait until the next month or quarter. As you free up time, consider integrating more detailed statements from additional sources to create a richer financial picture.

For instance, if you pay off your credit card from your main account every month, those lump-sum transactions might initially be recorded under expenses:credit card. However, if you begin receiving detailed statements from your credit card provider, you can refine your records. You might then assign transactions that pay off your balance to liabilities:credit card:balance payments, while processing the statement so that credit card spending is allocated to various expenses:<expense category> accounts. The credit card balance payments, as reported by credit card provider, would move cash from liabilities:credit card:balance payments to liabilities:credit card.

Once you have a system to process credit card statements, you can adjust the CSV conversion rules for your main account, regenerate all current account journals, and obtain a detailed breakdown of your credit card spending. The same strategy applies to Amazon, PayPal, your pension, savings, or brokerage accounts. Start by treating these as broad categories or black boxes, and gradually incorporate detailed statements. With everything under version control, it’s straightforward to make changes and verify that all aggregate reports adjust accordingly.

Separate source files, manual entries, and reports

The organisation that I ended up with is:

• Keep all journal files in a single directory with two subdirectories: one for the source files from financial institutions (./import), and one for generated summaries and reports (./export).

• At the top level there is all.journal which is the main entry point for all of your financial history. It contains just a bunch of include lines for your yearly files.

• Next to it are a bunch of {year}.journal files that will have:

  1. include of the opening balances file carrying over your assets/liabilities from the previous year;
  2. a bunch of includes for all the journals produced by converting the source CSV/PDF files;
  3. all manual transactions that you have for that year - you are not going to do manual entries anywhere else

• Raw CSV/PDF files go into ./import/{source}/in

• If they require some preprocessing into easier-to-consume CSV files, they will go into ./import/{source}/csv

• Converted CSV/PDF files go into ./import/{source}/journal

• Run top-level scrip (./export.sh) after each change to produce all the reports, check the changes to the reports generated in ./export, commit everything religiously.

• When you run hledger manually, point it to all.journal most of the time.

This is explained in greater details in the subsequent chapters.

Next steps

Head to Getting started and which explains how to grow this setup organically from scratch.