Spark Command: /opt/java/openjdk/bin/java -cp /opt/spark/conf:/opt/spark/jars/slf4j-api-2.0.17.jar:/opt/spark/jars/* -Dspark.master.ui.title=Prod Spark Cluster -Dspark.kubernetes.namespace=spark -Dspark.master.rest.enabled=true -Dspark.master.rest.host=0.0.0.0 -Dspark.ui.reverseProxy=true -Dspark.kubernetes.container.image=apache/spark:4.1.1-scala -Xmx1g -XX:+IgnoreUnrecognizedVMOptions --add-modules=jdk.incubator.vector --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED -Djdk.reflect.useDirectMethodHandle=false -Dio.netty.tryReflectionSetAccessible=true -Dio.netty.allocator.type=pooled -Dio.netty.handler.ssl.defaultEndpointVerificationAlgorithm=NONE --enable-native-access=ALL-UNNAMED org.apache.spark.deploy.master.Master --host prod-master-0 --port 7077 --webui-port 8080
========================================
WARNING: Using incubator modules: jdk.incubator.vector
Using Spark's default log4j profile: org/apache/spark/log4j2-defaults.properties
26/06/03 09:44:50 INFO Master: Started daemon with process name: 20@prod-master-0
26/06/03 09:44:50 INFO SignalUtils: Registering signal handler for TERM
26/06/03 09:44:50 INFO SignalUtils: Registering signal handler for HUP
26/06/03 09:44:50 INFO SignalUtils: Registering signal handler for INT
26/06/03 09:44:51 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j2-defaults.properties
26/06/03 09:44:52 INFO SecurityManager: Changing view acls to: spark
26/06/03 09:44:52 INFO SecurityManager: Changing modify acls to: spark
26/06/03 09:44:52 INFO SecurityManager: Changing view acls groups to: spark
26/06/03 09:44:52 INFO SecurityManager: Changing modify acls groups to: spark
26/06/03 09:44:52 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: spark groups with view permissions: EMPTY; users with modify permissions: spark; groups with modify permissions: EMPTY; RPC SSL disabled
26/06/03 09:44:53 INFO Utils: Successfully started service 'sparkMaster' on port 7077.
26/06/03 09:44:53 INFO Master: Starting Spark master at spark://prod-master-0:7077
26/06/03 09:44:53 INFO Master: Running Spark version 4.1.1
26/06/03 09:44:53 INFO JettyUtils: Start Jetty 0.0.0.0:8080 for MasterUI
26/06/03 09:44:53 INFO Utils: Successfully started service 'MasterUI' on port 8080.
26/06/03 09:44:54 INFO MasterWebUI: Bound MasterWebUI to 0.0.0.0, and started at http://prod-master-0:8080
26/06/03 09:44:55 INFO Master: Spark Master is acting as a reverse proxy. Master, Workers and Applications UIs are available at http://prod-master-0:8080
26/06/03 09:44:55 INFO Utils: Successfully started service on port 6066.
26/06/03 09:44:55 INFO StandaloneRestServer: Started REST server for submitting applications on 0.0.0.0 with port 6066
26/06/03 09:44:55 INFO Master: I have been elected leader! New state: ALIVE
26/06/03 09:45:05 INFO Master: Registering worker 192.168.139.187:36197 with 4 cores, 6.8 GiB RAM
26/06/03 09:45:11 INFO Master: Registering worker 192.168.139.146:36071 with 4 cores, 6.8 GiB RAM
26/06/03 09:45:24 INFO Master: Registering worker 192.168.139.169:43603 with 4 cores, 6.8 GiB RAM