+1 vote
19 views

This is kind of a continuation of this question: https://ask.openlca.org/918/fetches-from-collaboration-server-stopping

Software: Windows Server 2012, Tomcat 8.3. (xms2g, xmx4g), ElasticSearch 6.2.4 (xms8g, xmx10g), JDK 1.8.0_172

Hardware: Some virtual server, Intel Xeon, 16GB RAM

Starting from a fresh server start, we can fetch a repository (21,165 files in the repository folder, consisting of mostly reference data), the fetch can happen once. On the second, the fetch status will freeze and the openlca client hangs. On the server side, debug level tomcat stdout logs show that the elasticsearch queries were complete, but the localaccess log doesn't show the post from the openlca client (POST http://website:8080/lca-collaboration/ws/public/fetch/request/group/repository?download=false). This can happen such that one client on machine A can fetch, and a second client on machine B cannot fetch. It will happen if both fetches are done on the same machine (after restarting the client).

On another small repository (214 files in the repository folder), we can fetch once into an empty local database. We start a new, empty local database, and fetch again. On the third time, the fetch status will freeze and the openlca client hangs (all from the above). This is all done using the same machine after restarting the client (I haven't tested with multiple clients).

On another repository, which consists only of openlca reference data (i.e., we created a new database with complete reference data and commit that to a new repository). This database will not fetch at all into an empty local database after a restart.

The server will still respond to other requests - show data set info through the web server, will initiate other fetches, etc.

Stopping and starting both the elasticsearch and tomcat services in windows does not appear to be enough to allow the fetches to happen again - the server must be restarted. 

asked by (500 points)
commented by (5k points)
Does your elasticsearch keep running or does it stop during this? I ran into a problem this week when the elasticsearch was trying to dump memory to disk and we ran out of disc space because of it.
commented by (5k points)
Ah no, it does keep running, otherwise you would not be able to browse the repository on the website
commented by (500 points)
I will say regarding elasticsearch and going through the logs - it's not clear to me how elasticsearch results get used by the collaboration server during a fetch. More specifically, the search query will ask for anything matching a given commit, with a size parameter of 10,000 - I think this means that a maximum of 10,000 items will be returned on "a page". If there were 27,000 items associated with a commit, how does openLCA deal with that limited output or is the output only limited if you want data generated for a web page? I assume it must not matter since we can fetch a repository with those conditions at least once.

Please log in or register to answer this question.

...