Apache Spark is an open source, general-purpose distributed computing engine used for processing and analyzing a large amount of data. Hence, adding sequential and unique IDs to a Spark Dataframe is not very straight forward, because of distributed nature of it.
Like this:
Like Loading...
Elasticsearch : What, How and Why?
At its core, you can think of Elasticsearch as a server that can process JSON requests and give you back JSON data in near real-time.
Share this:
Like this:
Continue Reading
Nginx as reverse proxy and IP resolution
We know how reverse proxies work and how it masks the backend server. If needed, this blog explains it beautifully. NginX, is a web server that can also be used. read more…
Share this:
Like this:
Continue Reading
Simple and Powerful ReverseProxy in Go
In this article we will learn about reverse proxy, where to use it and how to implement it in Golang. A reverse proxy is a server that sits in front. read more…
Share this:
Like this:
Continue Reading
SQOOP : Data transfer between Hadoop and RDBMS
Apache Sqoop is a tool designed for efficiently transferring bulk data between Apache Hadoop and relational databases or mainframes.
Share this:
Like this:
Continue Reading
Generate Sequential and Unique IDs in a Spark Dataframe
Apache Spark is an open source, general-purpose distributed computing engine used for processing and analyzing a large amount of data. Hence, adding sequential and unique IDs to a Spark Dataframe is not very straight forward, because of distributed nature of it.
Share this:
Like this:
Continue Reading