Retire With 4M · Journal Entry #028

Taxes: The Silent Wealth Destroyer

If you aren't managing your tax liability, you are paying a 20-30% "stupidity tax" on your freedom. In early retirement, your tax strategy is your most important product.

The Largest Controllable Expense

In the corporate matrix, taxes are mostly automated. They are taken from your paycheck before you ever see the money. In early retirement, you are the tax manager. With a $4M portfolio, taxes are likely your largest annual expense—exceeding housing, food, and travel combined.

Managing Tax Drag is the difference between a portfolio that lasts 30 years and one that lasts forever. We use three primary levers to keep our money working for us, not the state.

1. Asset Location Optimization

We don't just care about what we own; we care about where we own it.

Taxable Brokerage

We keep our Growth Engine here (VTI/VOO). Long-term capital gains rates (0%, 15%, 20%) are much more favorable than ordinary income rates.

Tax-Advantaged (IRA/401k)

We keep high-turnover assets and "ordinary" dividend payers here. This shields the distributions from annual taxation, allowing for pure compounding.

The HSA

The "Triple Tax Threat." Contributions are deductible, growth is tax-free, and withdrawals for medical hits are tax-free. It is our ultimate late-life bucket.

2. Managing the Dividend Engine

Our Dividend Engine (~$1.25M) is engineered for Qualified Dividends . Qualified dividends are taxed at capital gains rates rather than higher ordinary income rates. This simple distinction can save us $20,000+ per year in taxes on our $10k/mo income flow.

Key Insight: $10,000 in qualified dividends is worth significantly more than $10,000 in W-2 salary because the tax "leakage" is much smaller.

3. The Global Tax Arbitrage

Moving to Asia adds a layer of complexity—and opportunity. As US citizens, we are taxed on global income, but we can utilize tools like the Foreign Earned Income Exclusion (FEIE) and the Foreign Tax Credit (FTC) to avoid double taxation.

By living in a different jurisdiction, we can often "choose" our tax bracket by timing our capital gains realizations and Roth conversions while our local Asia-based expenses are low. It is a form of regulatory arbitrage that maximizes our surplus margin.

Systemic Transparency: The Tax Ledger

We use our spreadsheets to track our "Estimated Tax Liability" in real-time. We don't wait for April to find out how much we owe. By projecting our annual dividends and growth harvests, we can make strategic adjustments in December—such as tax-loss harvesting—to hit our target tax rate.

Final Take

Taxes are the "Matrix" trying to reclaim a portion of your freedom. By being strategic, legal, and documented, you can significantly reduce their impact. At $4M, your goal is to be a Tax-Efficient Steward . Minimize the drag, protect the compounding, and keep the surplus for your active purpose and your heirs.

Back to top