We report a new method for X-ray density ligand fitting and refinement that is suitable for a wide variety of small-molecule ligands, including macrocycles. The approach (called “xGen”) augments a force field energy calculation with an electron density fitting restraint that yields an energy reward during the restrained conformational search. The resulting conformer pools balance goodness-of-fit with ligand strain. Real-space refinement from pre-existing ligand coordinates of 150 macrocycles resulted in occupancy-weighted conformational ensembles that exhibited low strain energy. The xGen ensembles improved upon electron density fit compared with the PDB reference coordinates without making use of atom-specific B-factors. Similarly, on nonmacrocycles, de novo fitting produced occupancy-weighted ensembles of many conformers that were generally better-quality density fits than the deposited primary/alternate conformational pairs. The results suggest ubiquitous low-energy ligand conformational ensembles in X-ray diffraction data and provide an alternative to using B-factors as model parameters.