2023

February

• Debug Python Without Debugger
  

  During my work which attempts to add a new compset to the E3SM model, I got an error from the create_case script. However, the error does not provide much useful information as:

2022

October

• Rename an E3SM case
  

  Renaming an E3SM case is desired when you want to retain the name for a rerun.

March

• Python module in VS code issue
  

  One of the issues I recently had is the import module error when testing the swaty python package.

2021

December

• Install GDAL in Gitpod
  

  Gitpod provides an easy way to test and share the whole Python package.

April

• VS code on cluster issue
  

  VS code extension sometimes crashes on HPC during remote development.

2020

December

• Set up VS Code with multiple Github accounts
  

  SSH Key

October

• Numpy mask for 3D array
  

  How to use np.where and mask on a 3D array? If I use normal operation, I received an error like this:

September

• Battle 33: Fortran index with MPI
  

  Recently I made some changes to H2SC model I developed. After the modification, the model crashed after some time step.

July

• Battle 32: VS code keyboard shortcut in VNC
  

  Somehow in the VNC environment, the “u” becomes unknown when I was trying to set up the shortcut.

• Battle 31: A flexible model configuration system
  

  CIME is a rather flexible case management system. When we design a system configuration subsystem, we want it to be flexible so we can set up a system without providing all the information. To do so, the case system should use a default configuration if possible.

June

• Battle 30: Naming system
  

  A typicl ELM history file variable has the following structure:

• Battle 29: Github issue
  

  This post is trying to resolve some common issues in my Github usage:

• Battle 28: redesign the plot libary
  

  I have developed a Python library for various projects. This library has many different features including the plotting functions. But I am not satisfied with the current strategy/structure of some components. For example, I somehow wrote a list of functions for a similar purpose. Now it is time to re-organize some functions.

• Battle 27: Rename Excel sheet name
  

  Here is what I got from an Excel sheet name, there are some unexpected special characteristics. So I designed a simple method to resolve them.

• Battle 26: Pandas Date
  

  The pandas date function seems to be very complex! For example:

May

• Battle 25: VTK visualization
  

  n my recent posts, I discussed how to generate 3D plots using MayaVi and PyVista, both of them works but not as great as I expected. MayaVi has issue with rendering using OpenCL PyVista does not expose enough APIs to control the plot So these are great tools to start with, but later on I decided to switch back to raw VTK. I used another package called PyeVTK (https://github.com/paulo-herrera/PyEVTK), it basically help you to export data into the VTK format.

• Battle 24: Finding the perfect camera postion
  

  The question is how we can find a perfect camera position when plotting a 3D VTK object.

April

• Battle 23: VS Code for HPC cluster
  

  VVS Code has become my go-to code editor since its release. Here I want to share some tips how I used it for HPC development.

• Battle 22: scatter plot with kde using matplotlib
  

  How to produce a scatter alike plot using Matplotlib without using SNS or other packages?

• Battle 21: Using NetCDF C++ API on cluster
  

  I have extended the capability of HexWatershed, which allows it to read MPAS mesh grid.

March

• Battle 20: QGIS failed to define spatial reference
  

  I need to convert a raster file to global scale. Earlier I manually resmapled it using Python SciPy, but the result is not very nice. Later on, I used the GDAL Python API, the results is great. However, when I tried to compared the two rasters, QGIS failed to define the projection.

February

• Battle 19: Depression filling on a sphere
  

  Question 1: How to break a land into several parts? Answer: The original solution does not restrict to single watershed: iWatershed = iWatershed + 1;

• Battle 18: IDL draw shapefile for HexWatershed results
  

  The HexWatershed model assumes that the spatial references of the DEM and Hexagon grid are the same. In reality, there are often not.

• Battle 17: IDL cgmap_grid cannot produce label
  

  There is a limit the increment of this function that it may take a long time to generate the label.

• Battle 16: IDL tiff issue
  

  EENVI function does not produce the GeoTiff by default. We need another function to convert it to GeoTiff. For example:

• Battle 15: IDL batch extraction
  

  I was using my own library to extract geotiff with shapefile, then the program stops after day 9.

• Battle 14: IDL cgplots over cgimage
  

  How can I draw a circle on top of a cgimage in IDL? This question is similar to this question:

• Battle 13: MPI auto resubmit
  
  • Prepare a simple checkpoint file;
  • Finish the main program, which only accept one parameter as input;
  • In the main program, check the range from checkpoint;
  • Finish the run_task function, which runs elemental task;
  • Prepare the entrance job script;
  • Submit the job.
• Battle 12: MPI QA
  

  I don’t alway have to deal with MPI myself, but I recently used it for several projects. I have several questions to share:

January

• Mpi4py on cluster
  

  One of my Python scripts is taking too long to run, which exceeds the short queue. I decided to use MPI, and mpi4py seems to be a good choice. But there are many issue when I first used it.

• Battle 10: USGS Python tool
  

  Lots of USGS data are distributed using web service. Earlier I have introduced how to download MODIS data in batch mode using wget. Recently I need to download USGS data in batch mode, so I collected some information I found online. In case someone might find it helpful.

• Battle 9: x2go VNC connection issue
  

  VNC is very convenient for many reasons. Somehow my x2go failed to work after recent playing around.

• Battle 8: Create a customized axis
  

  https://jakevdp.github.io/PythonDataScienceHandbook/04.08-multiple-subplots.html https://matplotlib.org/3.1.0/tutorials/toolkits/axes_grid.html

• Battle 7: create a perfect Y axis
  

  The question is: how to create a “perfect” Y axis for a time series plot.

2019

December

• Battle 6: define a DOI for each model simulation
  

  Simulation management can be tough if you run hundreds of simulation. The question is: how should we keep track of simulations within code?

• Battle 5: VS Code Python failed with OpenBLAS blas_thread_init: RLIMIT_NPROC
  

  When I was trying to debug a Python script on cluster, I received the following error: “OpenBLAS blas_thread_init: RLIMIT_NPROC”

• Battle 4: setting the right size for map plot
  

  How to control the size of text so they look pleasantly for readers?

November

• Battle 3: IDL cgmap grid frame
  

  IDL is so far my go-to programming language in terms of map visualization.

October

• Battle 2: Python submit SLURM job
  

  Programmers are supposed to be lazy, like Emacs user wants to do everything within Emacs because it can be an operating system.

• Battle 1: MODFLOW-NWT output LMT
  

  While I was trying to run the MT3D, I received an error: Error Reading Flow-Transport Link File Possibly Caused by:

  1. Incompatible Styles of Unformatted Files Used by MODFLOW and MT3D-USGS, ensure usage of FREE in MT3D-USGS name file matches LMT;
  2. Unformatted Flow-Transport Link File Saved by Version 1 of LinkMT3D Package Which Is No Longer Supported by MT3D-USGS.