Neurons and oligodendrocytes are terminally differentiated cells that sustain cascades of gene activation and repression to execute highly specialised functions, while retaining homeostatic control. To study long-range gene regulation, we developed Genome Architecture Mapping in combination with immunoselection (immunoGAM), and applied it to three cell types from the adult murine brain: oligodendroglia (OLGs) from the cortex, dopaminergic neurons (DNs) from the midbrain and pyramidal glutamatergic neurons (PGNs) from the hippocampus. We report extensive cell-type specialisation of 3D chromatin contacts across genomic scales, including predominant reorganisation of topological domains (TADs) and compartments. We also discover the large scale decondensation, or ‘melting’, of long genes when most highly expressed, many of which with roles in neurodevelopmental and neurodegeneration disorders. Through integration of 3D genome topology with expression and chromatin accessibility, we find that contacts most specific of DNs or PGNs contain genes associated with their specialised functions, such as addiction and synaptic plasticity, respectively, while harbouring putative binding sites for specific transcription factors. Lastly, sensory receptor genes occupy heterochromatic compartments which contact across tens of megabases in brain cells. Our results show that brain cells have highly specific 3D genome structure, which is tightly related with regulation mechanisms of genes with specialised functions.