The human immune system comprises various cell-type populations with major roles in preventing and combating pathogen infections, cancer progression, and ultimately maintaining the homeostasis of immunological responses. Here, we present the largest population-scale single-cell RNA sequence dataset from 1,267,758 peripheral blood mononuclear cells from 982 healthy human individuals reported to date. We show that the expression of a large number of genes is heritable in a context-specific manner across 16 populations of immune cells spanning the adaptive and innate lineages. Genes with large heritability estimates were associated with core immunological processes such antigen processing and presentation, cytotoxicity, cell activation, and cytokine signalling. Furthermore, we identified dynamic changes of gene expression heritability across the B cell and T cell landscape governing immunological processes such as memory and cell cytotoxicity. Interestingly, gene expression heritability correlated with open chromatin regions defined from single-cell data. Finally, we identified heritable genes associated with autoimmune disorders and immune traits. In summary, this study is the first one to describe genetic effects in gene expression in specific immune cell type compartments and improve the characterisation of the genetic architecture of immune gene expression and its implication in immune-mediated diseases.