Spatially Inhomogeneous Correlation Estimator for Temperature and Polarisation

by
Anthony Challinor (Astrophysics Group, Cavendish Lab, Cambridge, UK),
Gayoung Chon (LBL, Berkeley, USA),
Stéphane Colombi (Institut d'Astrophysique de Paris, France),
Eric Hivon (IPAC/Caltech, USA & IAP),
Simon Prunet (Univ. of Toronto, Canada & IAP),
István Szapudi (Institute for Astronomy, Hawaii, USA).

PolSpice (aka Spice) is a tool to statistically analyze Cosmic Microwave Background (CMB) data, as well as any other diffuse data pixelized on the sphere.

This Fortran90 program measures the 2 point auto (or cross-) correlation functions w(θ) and the angular auto- (or cross-) power spectra C(ℓ) from one or (two) sky map(s) of Stokes parameters (intensity I and linear polarisation Q and U). It is based on the fast Spherical Harmonic Transforms allowed by isolatitude pixelisations such as HEALPix [for Npix pixels over the whole sky, and a C(ℓ) computed up to ℓ=ℓmax, PolSpice complexity scales like Npix1/2 ℓmax2 instead of Npix ℓmax2]. It corrects for the effects of the masks and can deal with inhomogeneous weights given to the pixels of the map. In the case of polarised data, the mixing of the E and B modes due to the cut sky and pixel weights can be corrected for to provide an unbiased estimate of the "magnetic" (B) component of the polarisation power spectrum. Most of the code is parallelized for shared memory (SMP) architecture using OpenMP.

- Main input files are the HEALPix sky maps to analyse, and the pixel masks and weights if any. Those can be either in FITS binary format or in Planck-HFI internal I/O format/DMC database objects.
- Main input parameters are file names, and various options such as (gaussian) beam window correction, pixel window correction, noise substraction, angular range of the correlation functions and apodization angle.
- Output w(θ) and C(ℓ) are plain text or FITS files or DMC objects.
- empirical TT C(ℓ) covariance matrix can be produced.

Extra features:
- The maps can be either in RING or NESTED scheme. They can be in full-sky or cut-sky format.
- In the case of cross-correlation, the 2 maps being analyzed should have the same coordinates
  but can have been observed with different beams, and can have different pixel sizes (ie, different Nside, since v03-03-01).
  The mask and weighting scheme applied to each map should have the same Nside as the map to which they apply;
  they can be different for each map but should spatially overlap.
- A different pixel mask can be applied to the intensity and polarization components of the map(s).
- The best fit monopole and dipole can be regressed out of unmasked pixels before the analysis (since version 02-05-08)
- IDL routines (src/ispice.pro, src/read_spice.pro) are provided to run PolSpice and read its results from IDL or its free clones GDL or FL
- A python routine (src/ispice.py) is provided to run PolSpice from python 2 and 3.
- Standard Healpix/IDL and healpy tools can be used to visualize PolSpice outputs.
- For instance, IDL (bin_llcl.pro) and python (bin_llcl.py) routines are available to bin the output C(ℓ), as discussed here.
- If linked with Healpix 3.10 or above, arbitrary extensions and columns can be read from the input FITS files.

More details on the theory can be found in
Spice C(ℓ) Estimator:
-  Chon G. et al, 2004, MNRAS 350, 914
-  Szapudi, Prunet & Colombi, 2001, ApJ 548, 115
C(ℓ) Covariance Matrix:
-  Efstathiou G., 2004, MNRAS 349, 603
-  Challinor & Chon, 2005, MNRAS 360, 509
Other cut sky C(ℓ) estimators:
-  MASTER: Hivon E. et al, 2002, ApJ 567, 2
-  POKER: Ponthieu N. et al, 2011, A&A 535, A90
-  NAMASTER: Alonso D. et al, 2019, MNRAS, 484, 4127

Examples of PolSpice application to data analysis
CMB Science:
- BICEP 2yr data analysis: paper (Chiang et al, 2009; arXiv:0906.1181), data product
- CMBPol Mission Concept Study: Prospects for polarized foreground removal (Dunkley et al, 2008; arXiv:0811.3915)
- Planck 2013, 2015 and 2018 frequency maps (auto and cross) angular power spectra
Gravitational Lensing & Dark Matter:
- Dark Energy Survey data analysis: Cross-correlation of gravitational lensing from DES Science Verification data with SPT and Planck lensing
Neutrino & High Energy Particule Science:
- IceCube data analysis: Observation of Anisotropy in the Arrival Directions of Galactic Cosmic Rays at Multiple Angular Scales with IceCube

Requirements for installation:
- Fortran90 compiler (some are available for free for Unix, Linux, MacOSX and Windows, such as gfortran and g95),
- C compiler (since version v03-06-01),
- recent HEALPix Fortran90 library (3.60 or more required for v03-06-01 and above, 3.20 or more is OK for previous releases),
- cfitsio library (also required by HEALPix, which needs cfitsio 3.20 or more, 3.44 or more is recommended by cfitsio developers; version 4.1.0 or more is required for Fortran codes running on Apple's ARM chips).

Nice to have:
- Although not required, pkgconfig and/or CMake will make the compilation and installation easier.

Problems with PolSpice?
- Note 1: versions v02-08-00 and v03-00-01 require the flag -ffree-line-length-0 flag when compiled with gfortran
- README in html
- Known bugs
- Questions (and answers) on PolSpice usage and its results

Downloading Spice

Files available for download
   Version    Type       Date        Features
README.html Documentation 2024-09-18latest release documentation (also included in release tarball)
HISTORY History 2024-09-18code modification history (also included in release tarball)
INSTALL Installation 2024-09-18How to install the code (also included in release tarball)
v03-08-03 tar.gz archive 2024-09-18 latest release:
  • bug correction in bin_llcl.py
  • ispice.py has version number and is compatible with python 3.12
  • more information in output C(l) FITS header
  • fixes typos introduced in v03-08-02
v03-08-01 tar.gz archive 2023-12-22 previous release:
  • can now run directly on anafast C(l) produced by healpy;
  • fixed bugs appearing on ifort;
  • can deal with 9-column input C(l) (non-symmetric cross spectrum)
v03-07-05 tar.gz archive 2022-08-24 old release:
  • fixed a bug occuring when mixing objects in cut-sky and full-sky formats;
  • removed now deprecated arithmetic IFs from rec3jj routine;
with kudos to C. Daley, Y. Omori and A. Rahlin, for pointing the bug out.
v03-07-04 tar.gz archive 2022-05-13 old release: with kudos to C. Walter, W. Quan and A. Doussot for pointing those out.
v03-07-03 tar.gz archive 2021-06-04 old release:
- The aℓm coefficients of the analyzed map(s) can be weighted via the FITS file(s) alm1weightin and alm2weightin (and the resulting coefficients can still be dumped into alm1fileout and alm2fileout).   Note that doing this weighting will bias the output results.
- Improvements and bug fixes in ispice.py and bin_llcl.py
- Improved the CMake-based compilation, now requiring cmake 3.9+ (see INSTALL file)
v03-07-02 tar.gz archive 2020-11-06 old release:
- fixes a bug preventing the co-analysis of 2 maps with only of them masked (reported by E. Camphuis),
- ispice.py now accepts ~ and $ in input file names.
v03-07-01 tar.gz archive 2020-10-20 old release:
- fixes a bug affecting the alm dumped in the case of small maps (with fsky < 0.25)
v03-07-00 2020-10-16 old release:
- fixes several bugs, including one causing the code to crash when run as a command line (ie, without the IDL or python interfaces ispice.pro and ispice.py; reported by G. Addison and A. Zonca),
- expanded documentation of kernelsfileout (look at header of generated FITS file for more details).
v03-06-09 2020-09-22 old release:
- the aℓm coefficients of the (masked and/or weighted) map(s) can now be dumped into FITS file(s) thanks to the parameters alm1fileout and alm2fileout.
Those FITS files can be read in python with healpy.read_alm and in IDL with HEALPix's fits2alm.
v03-06-08 tar.gz archive 2020-05-26 old release:
- systematically sort partial sky maps, masks and weights on pixel index for easier combination and to detect duplicate pixels
v03-06-07 tar.gz archive 2020-05-22 old release:
- fixed segmentation fault in ClxCl covariance matrix (problem reported by Etienne Camphuis),
- added omp_num_threads, kernelsfileout, noiseclfile, noisecorfile and normfac to ispice.py
v03-06-06 tar.gz archive 2020-04-10 old release:
- maps, masks and weights can be a mixture of full-sky and partial-sky objects
- debugged parsing of input text C(l) files (both problems reported by Wei Quan)
v03-06-04 tar.gz archive 2020-02-04 old release:
- bug correction in mask and weight reading,
- made compliant with gfortran compiler (both problems reported by Loris Colombo)
v03-06-02 tar.gz archive 2020-01-30 old release:
- bug correction in reading mask(s) and/or weight(s) for polarisation,
- partial IQU FITS files, as generated by healpy, can be read,
- improved ispice.py
v03-06-01 2019-12-20 old release:
- requires HEALPix 3.60 (released Dec 2019) or more
- smaller memory footprint, especially for small maps (fsky < 25%) stored in cut-sky format
- requires a C compiler
v03-05-02 2019-06-06 old release:
- ispice.py now runs in python2 and python3 (kudos to J. Allingham)
v03-05-01 2018-01-16 old release:
- double precision compilation now proposed by default (see FAQ for details),
- a compilation with CMake now possible (see INSTALL document, in PolSpice top directory).
v03-04-01 2017-08-29 old release:
- much faster TT, EE, BB and TE kernels calculation with optimized Wigner 3J symbols recursion (based on routines provided by Franz Elsner),
- tenormfileout can now be used instead of windowfileout to save time and disk space on TE calculation (see FAQ for details),
- faster analysis of small patches by ignoring empty rings,
- reshuffled Makefile_template to accommodate new code and allow multi-job compilation (make -j).
v03-03-02 2017-04-07 old release:
- kernelsfileout now outputs apodization induced kernel even for Temperature only analysis (see FAQ for details)
v03-03-01 2017-02-21 old release:
- support for cross-correlation of maps with discrepant pixel sizes (Nside)
- introduction of python routine bin_llcl.py to bin output C(ℓ)
v03-02-00 2016-08-09old release:
* bug correction in reading of polarized cut-sky FITS maps (thanks to Tilman Troester);
* improvements to read_spice.pro:
  - documentation header,
  - deal with 9-column ASCII outputs;
* improvements to ispice.py by Kyle Story:
  - added corfile argument to output angular correlation functions,
  - more efficient search for spice executable.
v03-01-06 2015-10-12old release:
- when 2 polarized maps are cross-correlated, 9 or 6 spectra are computed, depending on symmetric_cl setting
- introduced tolerance keyword for E/B decoupling
- bugs correction in keyword parsing
- new python wrapper (ispice.py)
- bugs correction and improvements in IDL wrapper (ispice.pro)
- add SUBDIPOL in output FITS header
v03-00-03 2015-03-02old release:
- fixes bug with un-initialized map variables (addresses Nside≤32 problem described on cosmocoffee)
- all lines are now shorter than 132 characters (default gfortran limit)
HEALPix 3.20 or more recommended.
v03-00-01 2014-04-14old release:
- added listmapfiles*_* and listmapweights*_* options to allow linear combination of input maps
- computes correctly cross-power spectrum of 2 maps when the first one is masked or weighted (maskfile and/or weightfile are set), and the second one is not (maskfile2 and weightfile2 not set)
- turned-off monopole and dipole removal on (Q,U) maps in mask (and weight) free cases
- correct default value of extramap* in example parameter file (--help option)
See Note 1 above
v02-09-00 2013-03-29old release:
makes use of CFTISIO Extended File Name features available in HEALPix 3.10 and above, allowing the reading of arbitrary FITS extensions and columns in beam_file*, mapfile* and maskfile*.
HEALPix 3.10 or more recommended.

Back to PolSpice Main Page       Known Bugs        FAQ       

Last update for this page: 2024-09-18
Page maintained by: Eric Hivon (hivon at iap dot fr)