The NELAC Institute (TNI) is a 501(c)(3) non-profit organization whose mission is to foster the generation of environmental data of known and documented quality through an open, inclusive, and transparent process that is responsive to the needs of the community. The organization is managed by a Board of Directors and is governed by organizational Bylaws. Learn more...
One of the ways that TNI fosters the generation of data of known and documented quality is through the National Environmental Laboratory Accreditation Program, or NELAP. The purpose of this program is to establish and implement a program for the accreditation of environmental laboratories. Go to NELAP Home Page...
Ratio Master 1.7.5 11
Three Guidance Documents relating to Proficiency Testing Reporting Limits, Instrument Calibration, and Limit of Detection and Limit of Quantitation have been developed to assist with implementation of the 2016 Standard.
root@hadoop1:# spark-submit --class testesVitor.JavaWordCounter --master yarn sparkwordcount-0.0.1-SNAPSHOT.jar /user/vitor/Posts.xml 2 > output.txtSLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/usr/lib/spark/assembly/lib/spark-assembly-1.1.0-cdh5.2.0-hadoop2.5.0-cdh5.2.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: See _bindings for an explanation.SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]14/11/18 16:26:49 INFO SecurityManager: Changing view acls to: root14/11/18 16:26:49 INFO SecurityManager: Changing modify acls to: root14/11/18 16:26:49 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root)14/11/18 16:26:51 INFO Slf4jLogger: Slf4jLogger started14/11/18 16:26:51 INFO Remoting: Starting remoting14/11/18 16:26:52 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriver@hadoop1.example.com:58545]14/11/18 16:26:52 INFO Remoting: Remoting now listens on addresses: [akka.tcp://sparkDriver@hadoop1.example.com:58545]14/11/18 16:26:52 INFO Utils: Successfully started service 'sparkDriver' on port 58545.14/11/18 16:26:52 INFO SparkEnv: Registering MapOutputTracker14/11/18 16:26:52 INFO SparkEnv: Registering BlockManagerMaster14/11/18 16:26:52 INFO DiskBlockManager: Created local directory at /tmp/spark-local-20141118162652-0ff314/11/18 16:26:52 INFO Utils: Successfully started service 'Connection manager for block manager' on port 46763.14/11/18 16:26:52 INFO ConnectionManager: Bound socket to port 46763 with id = ConnectionManagerId(hadoop1.example.com,46763)14/11/18 16:26:52 INFO MemoryStore: MemoryStore started with capacity 267.3 MB14/11/18 16:26:52 INFO BlockManagerMaster: Trying to register BlockManager14/11/18 16:26:52 INFO BlockManagerMasterActor: Registering block manager hadoop1.example.com:46763 with 267.3 MB RAM14/11/18 16:26:52 INFO BlockManagerMaster: Registered BlockManager14/11/18 16:26:52 INFO HttpFileServer: HTTP File server directory is /tmp/spark-cfde3cf0-024a-47db-b97d-374710b989fc14/11/18 16:26:52 INFO HttpServer: Starting HTTP Server14/11/18 16:26:52 INFO Utils: Successfully started service 'HTTP file server' on port 40252.14/11/18 16:26:54 INFO Utils: Successfully started service 'SparkUI' on port 4040.14/11/18 16:26:54 INFO SparkUI: Started SparkUI at :404014/11/18 16:27:00 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable14/11/18 16:27:00 INFO EventLoggingListener: Logging events to hdfs://hadoop1.example.com:8020/user/spark/applicationHistory/spark-count-141633521799914/11/18 16:27:01 INFO SparkContext: Added JAR file:/root/sparkwordcount-0.0.1-SNAPSHOT.jar at :40252/jars/sparkwordcount-0.0.1-SNAPSHOT.jar with timestamp 141633522110314/11/18 16:27:01 INFO RMProxy: Connecting to ResourceManager at hadoop1.example.com/192.168.56.101:803214/11/18 16:27:02 INFO Client: Got cluster metric info from ResourceManager, number of NodeManagers: 314/11/18 16:27:02 INFO Client: Max mem capabililty of a single resource in this cluster 102914/11/18 16:27:02 INFO Client: Preparing Local resources14/11/18 16:27:02 INFO Client: Uploading file:/usr/lib/spark/assembly/lib/spark-assembly-1.1.0-cdh5.2.0-hadoop2.5.0-cdh5.2.0.jar to hdfs://hadoop1.example.com:8020/user/root/.sparkStaging/application_1415718283355_0004/spark-assembly-1.1.0-cdh5.2.0-hadoop2.5.0-cdh5.2.0.jar14/11/18 16:27:08 INFO Client: Prepared Local resources Map(__spark__.jar -> resource scheme: "hdfs" host: "hadoop1.example.com" port: 8020 file: "/user/root/.sparkStaging/application_1415718283355_0004/spark-assembly-1.1.0-cdh5.2.0-hadoop2.5.0-cdh5.2.0.jar" size: 95567637 timestamp: 1416335228534 type: FILE visibility: PRIVATE)14/11/18 16:27:08 INFO Client: Setting up the launch environment14/11/18 16:27:08 INFO Client: Setting up container launch context14/11/18 16:27:08 INFO Client: Yarn AM launch context:14/11/18 16:27:08 INFO Client: class: org.apache.spark.deploy.yarn.ExecutorLauncher14/11/18 16:27:08 INFO Client: env: Map(CLASSPATH -> $PWD:$PWD/__spark__.jar:$HADOOP_CLIENT_CONF_DIR:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOME/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*:$HADOOP_MAPRED_HOME/*:$HADOOP_MAPRED_HOME/lib/*:$MR2_CLASSPATH:$PWD/__app__.jar:$PWD/*, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 95567637, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1415718283355_0004/, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE, SPARK_USER -> root, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1416335228534, SPARK_YARN_CACHE_FILES -> hdfs://hadoop1.example.com:8020/user/root/.sparkStaging/application_1415718283355_0004/spark-assembly-1.1.0-cdh5.2.0-hadoop2.5.0-cdh5.2.0.jar#__spark__.jar)14/11/18 16:27:08 INFO Client: command: $JAVA_HOME/bin/java -server -Xmx512m -Djava.io.tmpdir=$PWD/tmp '-Dspark.tachyonStore.folderName=spark-ea602029-5871-4097-b72f-d2bd46c74054' '-Dspark.yarn.historyServer.address= :18088' '-Dspark.eventLog.enabled=true' '-Dspark.yarn.secondary.jars=' '-Dspark.driver.host=hadoop1.example.com' '-Dspark.driver.appUIHistoryAddress= :18088/history/spark-count-1416335217999' '-Dspark.app.name=Spark Count' '-Dspark.driver.appUIAddress=hadoop1.example.com:4040' '-Dspark.jars=file:/root/sparkwordcount-0.0.1-SNAPSHOT.jar' '-Dspark.fileserver.uri= :40252' '-Dspark.eventLog.dir=hdfs://hadoop1.example.com:8020/user/spark/applicationHistory' '-Dspark.master=yarn-client' '-Dspark.driver.port=58545' org.apache.spark.deploy.yarn.ExecutorLauncher --class 'notused' --jar null --arg 'hadoop1.example.com:58545' --executor-memory 1024 --executor-cores 1 --num-executors 2 1> /stdout 2> /stderr14/11/18 16:27:08 INFO SecurityManager: Changing view acls to: root14/11/18 16:27:08 INFO SecurityManager: Changing modify acls to: root14/11/18 16:27:08 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root)14/11/18 16:27:08 INFO Client: Submitting application to ResourceManager14/11/18 16:27:08 INFO YarnClientImpl: Submitted application application_1415718283355_000414/11/18 16:27:09 INFO YarnClientSchedulerBackend: Application report from ASM: appMasterRpcPort: -1 appStartTime: 1416335228936 yarnAppState: ACCEPTED14/11/18 16:27:10 INFO YarnClientSchedulerBackend: Application report from ASM: appMasterRpcPort: -1 appStartTime: 1416335228936 yarnAppState: ACCEPTED14/11/18 16:27:11 INFO YarnClientSchedulerBackend: Application report from ASM: appMasterRpcPort: -1 appStartTime: 1416335228936 yarnAppState: ACCEPTED14/11/18 16:27:12 INFO YarnClientSchedulerBackend: Application report from ASM: appMasterRpcPort: -1 appStartTime: 1416335228936 yarnAppState: ACCEPTED14/11/18 16:27:13 INFO YarnClientSchedulerBackend: Application report from ASM: appMasterRpcPort: -1 appStartTime: 1416335228936 yarnAppState: ACCEPTED14/11/18 16:27:14 INFO YarnClientSchedulerBackend: Application report from ASM: appMasterRpcPort: -1 appStartTime: 1416335228936 yarnAppState: ACCEPTED14/11/18 16:27:15 INFO YarnClientSchedulerBackend: Application report from ASM: appMasterRpcPort: -1 appStartTime: 1416335228936 yarnAppState: ACCEPTED14/11/18 16:27:16 INFO YarnClientSchedulerBackend: Application report from ASM: appMasterRpcPort: -1 appStartTime: 1416335228936 yarnAppState: ACCEPTED14/11/18 16:27:17 INFO YarnClientSchedulerBackend: Application report from ASM: appMasterRpcPort: -1 appStartTime: 1416335228936 yarnAppState: ACCEPTED14/11/18 16:27:18 INFO YarnClientSchedulerBackend: Application report from ASM: appMasterRpcPort: -1 appStartTime: 1416335228936 yarnAppState: ACCEPTED14/11/18 16:27:19 INFO YarnClientSchedulerBackend: Application report from ASM: appMasterRpcPort: -1 appStartTime: 1416335228936 yarnAppState: ACCEPTED14/11/18 16:27:20 INFO YarnClientSchedulerBackend: Application report from ASM: appMasterRpcPort: -1 appStartTime: 1416335228936 yarnAppState: ACCEPTED14/11/18 16:27:21 INFO YarnClientSchedulerBackend: Application report from ASM: appMasterRpcPort: -1 appStartTime: 1416335228936 yarnAppState: ACCEPTED14/11/18 16:27:22 INFO YarnClientSchedulerBackend: Application report from ASM: appMasterRpcPort: 0 appStartTime: 1416335228936 yarnAppState: RUNNING14/11/18 16:27:31 INFO YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after waiting maxRegisteredResourcesWaitingTime: 30000(ms)14/11/18 16:27:31 INFO MemoryStore: ensureFreeSpace(258371) called with curMem=0, maxMem=28024897514/11/18 16:27:31 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 252.3 KB, free 267.0 MB)14/11/18 16:27:31 INFO MemoryStore: ensureFreeSpace(20625) called with curMem=258371, maxMem=28024897514/11/18 16:27:31 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 20.1 KB, free 267.0 MB)14/11/18 16:27:31 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on hadoop1.example.com:46763 (size: 20.1 KB, free: 267.2 MB)14/11/18 16:27:31 INFO BlockManagerMaster: Updated info of block broadcast_0_piece014/11/18 16:27:31 INFO FileInputFormat: Total input paths to process : 114/11/18 16:27:31 INFO NetworkTopology: Adding a new node: /default/192.168.56.104:5001014/11/18 16:27:31 INFO NetworkTopology: Adding a new node: /default/192.168.56.103:5001014/11/18 16:27:31 INFO NetworkTopology: Adding a new node: /default/192.168.56.102:5001014/11/18 16:27:32 INFO SparkContext: Starting job: collect at JavaWordCounter.java:8414/11/18 16:27:32 INFO DAGScheduler: Registering RDD 3 (mapToPair at JavaWordCounter.java:30)14/11/18 16:27:32 INFO DAGScheduler: Registering RDD 7 (mapToPair at JavaWordCounter.java:68)14/11/18 16:27:32 INFO DAGScheduler: Got job 0 (collect at JavaWordCounter.java:84) with 228 output partitions (allowLocal=false)14/11/18 16:27:32 INFO DAGScheduler: Final stage: Stage 0(collect at JavaWordCounter.java:84)14/11/18 16:27:32 INFO DAGScheduler: Parents of final stage: List(Stage 2)14/11/18 16:27:32 INFO DAGScheduler: Missing parents: List(Stage 2)14/11/18 16:27:32 INFO DAGScheduler: Submitting Stage 1 (MappedRDD[3] at mapToPair at JavaWordCounter.java:30), which has no missing parents14/11/18 16:27:32 INFO MemoryStore: ensureFreeSpace(4096) called with curMem=278996, maxMem=28024897514/11/18 16:27:32 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 4.0 KB, free 267.0 MB)14/11/18 16:27:32 INFO MemoryStore: ensureFreeSpace(2457) called with curMem=283092, maxMem=28024897514/11/18 16:27:32 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 2.4 KB, free 267.0 MB)14/11/18 16:27:32 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on hadoop1.example.com:46763 (size: 2.4 KB, free: 267.2 MB)14/11/18 16:27:32 INFO BlockManagerMaster: Updated info of block broadcast_1_piece014/11/18 16:27:32 INFO DAGScheduler: Submitting 228 missing tasks from Stage 1 (MappedRDD[3] at mapToPair at JavaWordCounter.java:30)14/11/18 16:27:32 INFO YarnClientClusterScheduler: Adding task set 1.0 with 228 tasks14/11/18 16:27:32 INFO RackResolver: Resolved 192.168.56.104 to /default14/11/18 16:27:32 INFO RackResolver: Resolved 192.168.56.103 to /default14/11/18 16:27:32 INFO RackResolver: Resolved 192.168.56.102 to /default14/11/18 16:27:32 INFO RackResolver: Resolved hadoop2.example.com to /default14/11/18 16:27:32 INFO RackResolver: Resolved hadoop3.example.com to /default14/11/18 16:27:32 INFO RackResolver: Resolved hadoop4.example.com to /default14/11/18 16:27:36 ERROR YarnClientSchedulerBackend: Yarn application already ended: FAILED14/11/18 16:27:36 INFO SparkUI: Stopped Spark web UI at :404014/11/18 16:27:36 INFO DAGScheduler: Stopping DAGScheduler14/11/18 16:27:36 INFO YarnClientSchedulerBackend: Shutting down all executors14/11/18 16:27:36 INFO YarnClientSchedulerBackend: Asking each executor to shut down14/11/18 16:27:36 INFO YarnClientSchedulerBackend: Stopped14/11/18 16:27:36 INFO DAGScheduler: Failed to run collect at JavaWordCounter.java:84Exception in thread "main" org.apache.spark.SparkException: Job cancelled because SparkContext was shut down at org.apache.spark.scheduler.DAGScheduler$$anonfun$cleanUpAfterSchedulerStop$1.apply(DAGScheduler.scala:694) at org.apache.spark.scheduler.DAGScheduler$$anonfun$cleanUpAfterSchedulerStop$1.apply(DAGScheduler.scala:693) at scala.collection.mutable.HashSet.foreach(HashSet.scala:79) at org.apache.spark.scheduler.DAGScheduler.cleanUpAfterSchedulerStop(DAGScheduler.scala:693) at org.apache.spark.scheduler.DAGSchedulerEventProcessActor.postStop(DAGScheduler.scala:1399) at akka.actor.dungeon.FaultHandling$class.akka$actor$dungeon$FaultHandling$$finishTerminate(FaultHandling.scala:201) at akka.actor.dungeon.FaultHandling$class.terminate(FaultHandling.scala:163) at akka.actor.ActorCell.terminate(ActorCell.scala:338) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:431) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:447) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:262) at akka.dispatch.Mailbox.run(Mailbox.scala:218) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)root@hadoop1:# 2ff7e9595c
Comments