1、SuperVessel:EnablingSparkasaServicewithOpenStackandDockerGuanCheng(G.C.)ChenIBMResearch-Chinaweibo.com/parallellabsSuperVesselCloud•Public cloud built on the POWER7/POWER8 servers with OpenStack •It provides freeaccess for students, researchers, developers across the world, and helps grow O
2、penPOWER ecosystem (used in 30+ universiNes now) •It provides advanced technology services such as Spark as a Service, Docker Services, CogniNve CompuNng Service, IoT Service, Accelerator as a Service (FPGA and GPU) 4/18/15IBM Research ‐ China2 SparkasaServiceTry it at: www.ptopenlab.com3st
3、epstolaunchaSparkcluster,easy!Step 1: LoginStep 2: CreateStep 3: Ready!4/18/15IBM Research ‐ China3 WhyOpenStack?•Most popular IaaS soXware •Supports Docker* •Heat can orchestra docker containers easily – good for provision a Spark cluster Picture source: h-p://www.qyjohn.net/?p=38014/18/15
4、IBM Research ‐ China4 WhyDocker?•Less resource consumpNon than KVM –We can provision more Spark clusters! •Boot faster than KVM –Users like fast provision! •Incrementally build, revert and reuse your container –We love Git and AUFS! •However, Docker is not officially supported in OpenStack ye
5、t –Nova Docker is an external component of OpenStack•Port Docker to POWER Architecture (ppc64 and ppc64le)–Ubuntu 15.04 includes Docker for Picture source: h-p://www.zdnet.com/ar@cle/what‐is‐docker‐and‐why‐is‐it‐so‐darn‐popular/POWER8 ppc64le4/18/15IBM Research ‐ China5 WhySpark?•Fast •Unifi
6、ed •Ecosystem PorNng to POWER –Bugfix submieed to the community –Spark 1.3 works smoothly!4/18/15IBM Research ‐ China6 WhynotSahara?SaharaisacomponentforHadoop/SparkasaServiceinOpenStack•We started from OpenStack Icehouse … •DockerizaNon –Beeer service deployment and isolaNon for Big Data d
7、ashboard server •CustomizaNon –WaiNng for Sahara’s improvements is somehow *SLOW* •Docker, user analyNcs, Spark 1.4, Spark IDE, scheduling, data visualizaNon etc. 4/18/15IBM Research ‐ China7 ArchitectureDesignSparkMasterNameNodeBilling&AuthGlanceSparkDr
