A few days ago my Portal for ArcGIS server ran out of disk space. After extending the drive on the virtual machine Portal for ArcGIS wouldnt start.
The ArcGISPortal.exe was there but no other processes were spawned – like none at all – no postgres, no java, no javaw etc
Usually the culprit is the Postgres database. So step 1 is to try and kick start the database. You can do this by doing the following;
- Open a command prompt (but start it as the service account that runs Portal)
- Run the following command (adjust directories as necessary);
- CMD /C “”C:/ArcGIS/Portal/framework/runtime/pgsql/bin/postgres.exe” -D “C:/arcgisportal/db” -p 7654 < “nul” >> “C:\arcgisportal\logs\\database\pgsql.log” 2>&1″
If that works you should see a bunch of postgres.exe processes start up. You can also check the database logs to make sure its up and accepting connections – look at the latest file under C:\arcgisportal\logs\database\pg_log it should show “database system is ready to accept connections”.
In my case that worked but there were no additional processes spawned, so it wasnt the Postgres database. BTW you can press ctrl+c to stop the database in the command prompt.
During normal operation of Portal you will see two Java processes; java.exe and javaw.exe
The next step is to try and kick start those which is a bit more complicated but its similar to launching postgres process.
This is where you should probably contact Esri support BTW.
So for the java process you need to launch something like this;
“C:\ArcGIS\Portal\framework\runtime\jre\bin\java.exe” -Xms2048m -Xmx2048m -XX:CMSInitiatingOccupancyFraction=75 -XX:-UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow
-Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.io.tmpdir=C:/arcgisportal/dsdata/temp -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data
-XX:ErrorFile=logs/hs_err_pid%p.log -XX:+UseParallelGC -Djava.locale.providers=COMPAT -XX:UseAVX=2 -Dlog4j2.disable.jmx=true -Djava.io.tmpdir=C:/arcgisportal/dsdata/temp -Dio.netty.allocator.type=pooled -XX:MaxDirectMemorySize=1073741824 -Delasticsearch
-Des.path.home=”C:\ArcGIS\Portal\framework\runtime\ds\framework\runtime\elasticsearch” -Des.path.conf=”C:\ArcGIS\Portal\framework\runtime\ds\framework\runtime\elasticsearch\config” -Des.distribution.flavor=”oss” -Des.distribution.type=”zip” -Des.bundled_jdk=”true” -cp
“C:\ArcGIS\Portal\framework\runtime\ds\framework\runtime\elasticsearch\lib*” “org.elasticsearch.bootstrap.Elasticsearch” -d -p “C:\ArcGIS\Portal\framework\etc\pids\elastic.pid”
See not so easy…
Anyway that worked fine so lets try the next the javaw process;
“C:\ArcGIS\Portal\/framework/runtime/jre\bin\javaw” -Dportal=true -Dprofile=portal -Dspring.profiles.active=indexserver -XX:+UseParallelGC -XX:MinHeapFreeRatio=40 -XX:MaxHeapFreeRatio=70 “-Djdk.tls.ephemeralDHKeySize=2048”
-Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Djava.util.logging.config.file=”C:\ArcGIS\Portal\framework\runtime\tomcat\conf\logging.properties” -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dsun.locale.formatasdefault=true
-Dfile.encoding=UTF8 -Dclient.encoding.override.UTF8 -classpath “C:\ArcGIS\Portal\framework\runtime\tomcat\bin\bootstrap.jar;C:\ArcGIS\Portal\framework\runtime\tomcat\bin\tomcat-juli.jar” -Dcatalina.base=”C:\ArcGIS\Portal\framework\runtime\tomcat”
-Dcatalina.home=”C:\ArcGIS\Portal\framework\runtime\tomcat” org.apache.catalina.startup.Bootstrap start
That one didnt do anything.
So we now know Postgres is okay, java.exe is okay and javaw.exe is NOT okay.
You’ll notice that the command line running javaw mentions Tomcat so the next stop is to look for the Tomcat logs which are under; C:\ArcGIS\Portal\framework\runtime\tomcat\logs
In the log files it mentioned that the server.xml was malformed – this file lives in C:\ArcGIS\Portal\framework\runtime\tomcat\conf – this file is used to configure the Tomcat web server.
Looking at the file, it was zero kb i.e. empty.
The lack of diskspace had caused the file to not be written correctly (Portal updates the file when it shuts down BTW).
Luckily I could grab a file from a similar install and use that to correct the file.
Sorted? Well not quite.
The service started up and all processes were there but the web server was not available (ie no REST interface). Requests to 7080 would be forwarded to 7443 which would then not do anything, so it was there but SSL wasnt working. Weird.
Looking back through the server.xml file there is a section that has the SSL config and then I spotted the issue. The portal I had copied the server.cml file from had a different certificate alias set – the default self-signed certificate for Portal is has an alias of “portal”.
Stopped the service, updated the alias and started the service.
Back in action!
So moral of the story, dont run out of diskspace and if you do dont stop Portal before you have cleared out some space.