Resource Management Improvements

Performance and Resource Management Improvements

The server, when handling multiple requests and given the size of polygons, can take a long time for calculations. Since it uses all CPU cores, it consumes a lot of resources, potentially making the server unresponsive.

Changes:

• Timeout Handling:

  • Previously, timeout was handled from clojure/libpython-clj with future, but it wasn't always effective.
  • Moved the timeout functionality to the Python side.
  • If a calculation takes more than 50 seconds, it times out and frees resources.

• Polygon Size Limit:

  • Added an upper limit size for polygons, currently set at 10000.00 ha.
  • This avoids running costly, resource-hungry calculations.

• Concurrency Control:

  • Implemented a semaphore mechanism to limit multiple resource-hungry concurrent calls to stats calculations.
  • Currently set at 2 to balance between resource utilization and request handling capacity.

• Quick Patch:

  • Set loader timeout to 55 seconds, slightly longer than the calculation timeout, as sometimes the value doesn't get read from config.
  • This suffices for now since we timeout calculations at 50 seconds and HTTP requests time out at 1 minute.