Zeppelin Jira

Posted : admin | On 30-04-2021



Giants

This post summarizes the steps to install Zeppelin 0.7.3 in Windows environment.

Tools and Environment

  • GIT Bash
  • Command Prompt
  • Windows 10

Download Binary Package

Download the latest binary package from the following website:

Zeppelin

Atlassian Jira Project Management Software (v8.3.4#803005-sha1:1f96e09) About Jira; Report a problem; Powered by a free Atlassian Jira open source license for Apache Software Foundation. Try Jira - bug tracking software for your team. Zeppelin; ZEPPELIN-3633; ZeppelinContext Not Found in yarn-cluster Mode. . ZEPPELIN-3062 Removes ctrl+s default behavior from notebook ### What is this PR for? As a programmer I habitually press CTRL + S out of fear for my sanity.

In my case, I am saving the file to folder: F:DataAnalytics

Zeppelin Jar

UnZip Binary Package

Open Git Bash, and change directory (cd) to the folder where you save the binary package and then unzip:

$ cd F:DataAnalytics

fahao@Raymond-Alienware MINGW64 /f/DataAnalytics
$ tar -xvzf zeppelin-0.7.3-bin-all.gz

After running the above commands, the package is unzip to folder: F:DataAnalyticszeppelin-0.7.3-bin-all

Run Zeppelin

Before starting Zeppelin, make sure JAVA_HOME environment variable is set.

JAVA_HOME environment variable

JAVA_HOME environment variable value should be your Java JRE path.

Start Zeppelin

Run the following command in Command Prompt (Remember to the path to your own Zeppelin folder):

cd /D F:DataAnalyticszeppelin-0.7.3-bin-allbin

F:DataAnalyticszeppelin-0.7.3-bin-allbin>zeppelin.cmd

Wait until Zeppelin server is started:

Verify

In any of your browser, navigate to http://localhost:8080/

The UI should looks like the following screenshot:

Create Notebook

Create a simple note using markdown and then run it:

java.lang.NullPointerException

If you got this error when using Spark as interpreter, please refer to the following pages for details:

Basically, even you configure Spark interpreter not to use Hive, Zeppelin is still trying to locate winutil.exe through environment variable HADOOP_HOME.

Thus to resolve the problem, you need to install Hadoop in your local system and then add one environment variable:

After the environment variable is added, please restart the whole Zeppelin server and then you should be able to run Spark successfully.

You should also be able to run the tutorials provided as part of the installation:

org.apache.zeppelin.interpreter.InterpreterException:

Zeppelin Jira

If you encounter the following error:

org.apache.zeppelin.interpreter.InterpreterException: The filename, directory name, or volume label syntax is incorrect.

at org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess.start(RemoteInterpreterManagedProcess.java:143) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.reference(RemoteInterpreterProcess.java:73) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:265) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:430) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:111) at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:387) at org.apache.zeppelin.scheduler.Job.run(Job.java:175) at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:329) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

It is probably caused by the same issue in this JIRA task if you have installed Spark locally:

Jira

To fix it, you can remove ‘SPARK_HOME’ environment variable and your Spark should still be able to run correctly if you run spark shell using full path of spark-shell.cmd.

Skip to end of metadataGo to start of metadata

Apache Zeppelin is a web-based notebook that enables interactive data analytics while Apache Pig is a platform for analyzing large data sets that consists of a high-level language for expressing data analysis programs. Pig-latin is a very powerful languages for data flow processing.

One drawback of pig community complains about is that pig-latin is not a standard language like sql so very few BI tools integrate with it. So it is pretty to hard to visualize the result from pig. Now the good news is that Pig is integrated in zeppelin 0.7 where you can write pig latin and visualize the result.

Pig interpreter is supported from zeppelin 0.7.0, so first you need to install zeppelin, you can refer this link for how to install and start zeppelin.

Zeppelin supports 2 kinds of pig interpreters for now.

  • %pig (default interpreter)
  • %pig.query

%pig is like the pig grunt shell. Anything you can run in pig grunt shell can be run in %pig.script interpreter, it is used for running pig script where you don’t need to visualize the data, it is suitable for data munging. %pig.query is a little different compared with %pig.script. It is used for exploratory data analysis by using pig latin where you can leverage zeppelin’s visualization ability. There're 2 minor differences in the last statement between %pig and %pig.query

Zeppelin Jailbreak

  • No pig alias in the last statement in %pig.query (read the examples below).
  • The last statement must be in single line in %pig.query

Here I will give 4 simple examples to illustrate how to use these 2 interpreters. These 4 examples are another implementation of zeppelin tutorial where spark is used. We just do the same thing by using pig instead.

This script do the data preprocessing


Get the number of each age where age is less than 30


The same as above, but use dynamic text form so that use can specify the variable maxAge in textbox. (See screenshot below). Dynamic form is a very cool feature of zeppelin, you can refer this link for details.

Get the number of each age for specific marital type, also use dynamic form here. User can choose the marital type in the dropdown list (see screenshot below).

The following is a screenshot of these 4 examples. You can also check pig tutorial note which contains all the code of this blog in zeppelin.

Zeppelin Java

Pig interpreter in zeppelin supports all the execution engine that pig supports.

  • Local Mode
    • Nothing needs to be done for local mode
  • MapReduce Mode
    • HADOOP_CONF_DIR needs to be specified in ZEPPELIN_CONF_DIR/zeppelin-env.sh
  • Tez Local Mode
    • Nothing needs to be done for tez local mode
  • Tez Mode
    • HADOOP_CONF_DIR and TEZ_CONF_DIR needs to be specified in ZEPPELIN_CONF_DIR/zeppelin-env.sh

The default mode is mapreduce, but you can change that in interpreter setting. You can also set any pig configuration in the interpreter setting page. Here's one screenshot of that.

Zeppelin Girl Band

This is the first phase work to integrate pig into zeppelin. There’s lots of work needs to do in future. Here’s my current to-do list

  • Integrate spark engine so that we can use spark sql together with pig-latin
  • Integrate spark mllib so that we can use pig-latin to do machine learning
  • Add new interpreter %pig.udf to allow user to write java udf in zeppelin
  • Integrate more closely with datafu

Zeplin Jira Integration

If you have any other new ideas, please contact me at zjffdu@apache.org

Zeppelin Japan

or you can file ticket in apache zeppelin jira https://issues.apache.org/jira/browse/ZEPPELIN