Which of following is NOT an advantage of using structured programming with SparkSQL dataframes compared to programming using the Spark RDD API?Question 4Answera.Structured programming allows the use of a more optimised data layout which benefits CPU cache utilisation.b.Structure programming allows the system to use more optimised Java byte code when executing built-in functions.c.Structured programming allows the system to automatically perform query optimisation.d.Structured programming allows data to be cached in RAM.

Which of the following statements is false?Question 1Answera.Executing queries using SparkSQL Dataframes and DataSets functions are at least as fast as using their RDD counterparts, often faster.b.You can add columns to a dataframe using the withColumn function.c.After performing a self-join on a dataframe the resulting columns will contain duplicate column names.d.DataSets contain schemas whereas DataFrames do not contain schemas.

1.Question 1What are the three main components of Apache Spark architecture?1 pointScala; Java; PythonData; compute interface; resource managementStorage; HDFS; PythonMesos; YARN; Kubernetes2.Question 2What are DataFrames in Apache Spark?1 pointDataFrames is a distributed file system in Spark used for storing large data sets efficiently.DataFrames are a distributed collection of data organized into named columns.DataFrames are Spark’s built-in machine learning models for predictive analytics.DataFrames is a data format for storing graph data structures in Spark.3.Question 3What is Apache Spark?1 pointHardware manufacturerIn-memory framework for distributed data processingCloud storage serviceClosed-source data analysis tool4.Question 4What is functional programming?1 pointA programming approach that emphasizes the how to of the solution as opposed to the what of the solutionA programming approach that focuses solely on graphical functions and visual designs A programming method that prioritizes procedural programming over the use of mathematical functionsA style of programming that follows the mathematical function format5.Question 5Which of the following statements defines Resilient Distributed Datasets (RDDs)? Select all that apply.1 pointRDD is a collection of fault-tolerant elements.RDD is capable of receiving parallel operations.RDDs are immutable.RDD is a distributed database management system.6.Question 6What is the primary purpose of parallel programming?1 pointTo employ specific control and coordination mechanismTo run noncontemporary instructionsTo use multiple compute resources to solve a computational problemTo break a problem into discrete parts that can be solved sequentially7.Question 7Which of the following is a benefit of DataFrames?1 pointTo scale from kilobytes of data on multiple laptops to petabytes on a large clusterTo scale small-scale data on a laptopSupports specific data formats and storage systemsTo scale from kilobytes of data on a single laptop to petabytes on a large cluster

1.Question 1Which API does Apache Spark Structured Streaming use for processing streaming data? 1 pointDataFrame and Dataset APIs Kafka Streaming API Spark Streaming API Spark SQL API 2.Question 2Which output mode in Spark Structured Streaming is particularly useful for managing late-arriving data points? 1 pointOverwrite mode Append mode Update mode Complete mode 3.Question 3Which Spark feature allows you to query a dataframe? 1 pointPipeline SparkSQL DataFrame RDD 4.Question 4Which transformer/extractor counts the occurrences of each term in the text and constructs a vector representation? 1 pointStringIndexer StopWordsRemover StandardScaler CountVectorizer 5.Question 5In a certain SparkML pipeline, there are these 3 stages StandardScaler, VectorAssembler, and Linear Regression. Which of the following is the correct order? 1 pointVectorAssembler, StandardScaler, Linear Regression Linear Regression, StandardScaler, VectorAssembler StandardScaler, Linear Regression, VectorAssembler StandardScaler, VectorAssembler, Linear Regression 6.Question 6In which phase of the ETL are you likely to encounter "save data to parquet file"? 1 pointLoad Model Building Transform Extract 7.Question 7What does a StringIndexer do? 1 pointConverts categorical string columns into numerical indices. Converts numerical columns into strings. Indexes strings so that they can be accessed quicker. Converts floating point columns into string indices. 8.Question 8What does a Tokenizer do? 1 pointConverts text into words. Converts text into symbols called tokens. Assigns a token to each row of the dataset. Is used to compress text. 9.Question 9Which Pyspark component is used to load a stored model? 1 pointPipelineModel Pipeline Model ModelLoader 10.Question 10In which phase of the ETL process would you typically perform "data validation"? 1 pointLoad Extract Model Building Transform

Why use Apache Spark?

The three components of Spark architecture are: