Inequality has become a central policy issue across the world. We study trends of inequality in earnings, income and wealth across households in Japan, using the National Survey of Family Income and Expenditure (NSFIE) from 1984 to 2014. We focus on the transition of inequality unconditionally and conditionally across various dimensions of household heterogeneity such as age, cohort, employment and marital status of household heads, sources of income, family size, etc. Inequality in earnings, income and wealth all increased during the last three decades. Changes in earnings and income inequality were mostly driven by demographic shift in the population towards the elderly, who tend to have higher inequality. Wealth inequality rose not only in the aggregate but also among the young, and this is due to a major increase in the fraction of households who own zero or very low wealth across all age groups. Critical factors in understanding inequality trends in Japan that we identified are aging demographics, changes in typical household structure, and macroeconomic trends of the past decades including the financial bubble period and a decades-long slow-down thereafter.