add loadpremadeflags

[yoshinori-0778]

This PR adds new class of preprocess, which loads flags that made before and stored in aman. This is needed for running simulation of planet mapmaker that Sam and I have developed because it needs flags for a simlated planet that cannot be calculated in current preprocess. This is a code in pwg-script corresponding to this update.
Current way is

1. load and run some basic preprocess (e.g., pointing_model/fp_flags)
2. calculate and add simulated TOD into aman.signal, and calculate flags for the simlated planet.
3. Run rest of preprocess that run for actual planet mapmaking.
   Since the flags for the simlated planet is needed for these preprocess (e.g., PCA filter) in step 3, I would like to add this.

There could be more smart way, so please let me know if you come up with more smart way.

[msilvafe]

Can you please point me to an example config file and bash run script which you use to run the referenced script in pwg-scripts? Is this basically just used to cut on an existing flag? Why do you need to move that flag into proc_aman and save it?

[yoshinori-0778]

This is the example config file.
And you can run the referenced script in pwg-script by python3 sim.py --test under pwg-script/flp/planet_mapmaker. You have to chenge config path. I am testing in tiger3, so you also have to paths inside the yaml file above if you will test somewhere else.

This is mainly used for combined mask later part of preprocess, so I think we need to move flag into proc_aman. (But I might be wrong, and there is a better way) For example, we would like to use it as PCA mask.

- name: "combine_flags"
      process:
        flag_labels: ['turnaround_flags.turnarounds', 'sim_jupiter_flag.sim_jupiter_flag', 'jumps_slow.jump_flag', 'jumps_2pi.jump_flag']
        method: '+'
        total_flags_label: 'pca_exclude'

Thanks!

[msilvafe]

This is the example config file. And you can run the referenced script in pwg-script by python3 sim.py --test under pwg-script/flp/planet_mapmaker. You have to chenge config path. I am testing in tiger3, so you also have to paths inside the yaml file above if you will test somewhere else.

This is mainly used for combined mask later part of preprocess, so I think we need to move flag into proc_aman. (But I might be wrong, and there is a better way) For example, we would like to use it as PCA mask.

- name: "combine_flags"
      process:
        flag_labels: ['turnaround_flags.turnarounds', 'sim_jupiter_flag.sim_jupiter_flag', 'jumps_slow.jump_flag', 'jumps_2pi.jump_flag']
        method: '+'
        total_flags_label: 'pca_exclude'

Thanks!

Could we just modify the combine_flags function to look for flags in either the proc_aman or aman. For example if you had:
flag_labels: ['preprocess.turnaround_flags.turnarounds', 'sim_jupiter_flag.sim_jupiter_flag', 'preprocess.jumps_slow.jump_flag', 'preprocess.jumps_2pi.jump_flag']

Then in the combine_flags function we can add the logic if the flag name starts with preprocess it looks in proc_aman otherwise it looks in aman? Would that solve your problem?

[yoshinori-0778]

Thanks for the comment, let me think and try that.

[msilvafe]

This will not save your flags in the preprocess h5 archive. If you would like to do that then probably your initial approach is preferable. Potentially it could be extended to a more general process called "store_fields" or something similar which could be used to move arbitrary fields from aman to proc_aman to save in the h5 archive.

[yoshinori-0778]

Planet mapmaker have not saved the flags in the same way as CMB pipeline, but I would like to have that in the future. So can I proceed my initial approach? and yes, I think this preprocess has a expandability in the future, but I would like to leave its extention now.

[yoshinori-0778]

@msilvafe Do you have any further comment/question/request?

[msilvafe]

Let's just add some description in the docstring for the LoadPremadeFlags class about what this is used for (planet mapmaking) and how its typically used but otherwise based on the discussion in this PR it seems most sensible to leave it as is for now. Thanks.