Studies have shown that incorporating measurement-derived point spread functions (PSFs) in reconstruction significantly improves the image quality of positron emission tomography (PET). However, measuring the complete image grids and storing the whole system model are not feasible in practice. We measured the PSFs on a sparse grid and used a parameterization technique to estimate the PSFs at locations that were not measured. Symmetries of the PET scanner and a factorized matrix approach were used to address the storage problem. Our rotator-based algorithm was used to exploit radial symmetries on rectangular voxels. The overall system responses were factorized as a product of a geometric component and a projection space blurring component. The geometric component was also used to determine the axial peak locations of the measured PSFs. An initial implementation of our method was presented, which yielded a compact fully 3-D system model (less than 250 MB). The image quality in terms of resolution (>1300%) and contrast noise trade-offs (> 15%) was considerably improved compared to the reconstruction of the scanner’s standard algorithm.