# Android tutorial 2: A running pipeline

### Goal

The tutorials seen in the Basic and Playback sections are intended for Desktop platforms and, therefore, their main thread is allowed to block (using `gst_bus_pop_filtered()`) or relinquish control to a GLib main loop. On Android this would lead to the application being tagged as non-responsive and probably closed.

This tutorial shows how to overcome this problem. In particular, we will learn:

- How to move the native code to its own thread
- How to allow threads created from C code to communicate with Java
- How to access Java code from C
- How to allocate a `CustomData` structure from C and have Java host it

### Introduction

When using a Graphical User Interface (UI), if the application waits for GStreamer calls to complete the user experience will suffer. The usual approach, with the GTK+ toolkit for example, is to relinquish control to a GLib `GMainLoop` and let it control the events coming from the UI or GStreamer.

This approach can be very cumbersome when GStreamer and the Android UI communicate through the JNI interface, so we take a cleaner route: We use a GLib main loop, and move it to its own thread, so it does not block the application. This simplifies the GStreamer-Android integration, and we only need to worry about a few inter-process synchronization bits, which are detailed in this tutorial.

Additionally, this tutorial shows how to obtain, from any thread, the JNI Environment pointer required to make JNI calls. This is necessary, for example, to call Java code from callbacks in threads spawned deep within GStreamer, which never received this pointer directly.

Finally, this tutorial explains how to call Java methods from native C code, which involves locating the desired method’s ID in the class. These IDs never change, so they are cached as global variables in the C code and obtained in the static initializer of the class.

The code below builds a pipeline with an `audiotestsrc` and an `autoaudiosink` (it plays an audible tone). Two buttons in the UI allow setting the pipeline to PLAYING or PAUSED. A TextView in the UI shows messages sent from the C code (for errors and state changes).

### A pipeline on Android [Java code]

src/org/freedesktop/gstreamer/tutorials/tutorial_2/Tutorial2.java

    package org.freedesktop.gstreamer.tutorials.tutorial_2;

    import android.app.Activity;
    import android.os.Bundle;
    import android.util.Log;
    import android.view.View;
    import android.view.View.OnClickListener;
    import android.widget.ImageButton;
    import android.widget.TextView;
    import android.widget.Toast;

    import org.freedesktop.gstreamer.GStreamer;

    public class Tutorial2 extends Activity {
        private native void nativeInit();     // Initialize native code, build pipeline, etc
        private native void nativeFinalize(); // Destroy pipeline and shutdown native code
        private native void nativePlay();     // Set pipeline to PLAYING
        private native void nativePause();    // Set pipeline to PAUSED
        private static native boolean nativeClassInit(); // Initialize native class: cache Method IDs for callbacks
        private long native_custom_data;      // Native code will use this to keep private data

        private boolean is_playing_desired;   // Whether the user asked to go to PLAYING

        // Called when the activity is first created.
        @Override
        public void onCreate(Bundle savedInstanceState)
        {
            super.onCreate(savedInstanceState);

            // Initialize GStreamer and warn if it fails
            try {
                GStreamer.init(this);
            } catch (Exception e) {
                Toast.makeText(this, e.getMessage(), Toast.LENGTH_LONG).show();
                finish();
                return;
            }

            setContentView(R.layout.main);

            ImageButton play = (ImageButton) this.findViewById(R.id.button_play);
            play.setOnClickListener(new OnClickListener() {
                public void onClick(View v) {
                    is_playing_desired = true;
                    nativePlay();
                }
            });

            ImageButton pause = (ImageButton) this.findViewById(R.id.button_stop);
            pause.setOnClickListener(new OnClickListener() {
                public void onClick(View v) {
                    is_playing_desired = false;
                    nativePause();
                }
            });

            if (savedInstanceState != null) {
                is_playing_desired = savedInstanceState.getBoolean("playing");
                Log.i ("GStreamer", "Activity created. Saved state is playing:" + is_playing_desired);
            } else {
                is_playing_desired = false;
                Log.i ("GStreamer", "Activity created. There is no saved state, playing: false");
            }

            // Start with disabled buttons, until native code is initialized
            this.findViewById(R.id.button_play).setEnabled(false);
            this.findViewById(R.id.button_stop).setEnabled(false);

            nativeInit();
        }

        protected void onSaveInstanceState (Bundle outState) {
            Log.d ("GStreamer", "Saving state, playing:" + is_playing_desired);
            outState.putBoolean("playing", is_playing_desired);
        }

        protected void onDestroy() {
            nativeFinalize();
            super.onDestroy();
        }

        // Called from native code. This sets the content of the TextView from the UI thread.
        private void setMessage(final String message) {
            final TextView tv = (TextView) this.findViewById(R.id.textview_message);
            runOnUiThread (new Runnable() {
              public void run() {
                tv.setText(message);
              }
            });
        }

        // Called from native code. Native code calls this once it has created its pipeline and
        // the main loop is running, so it is ready to accept commands.
        private void onGStreamerInitialized () {
            Log.i ("GStreamer", "Gst initialized. Restoring state, playing:" + is_playing_desired);
            // Restore previous playing state
            if (is_playing_desired) {
                nativePlay();
            } else {
                nativePause();
            }

            // Re-enable buttons, now that GStreamer is initialized
            final Activity activity = this;
            runOnUiThread(new Runnable() {
                public void run() {
                    activity.findViewById(R.id.button_play).setEnabled(true);
                    activity.findViewById(R.id.button_stop).setEnabled(true);
                }
            });
        }

        static {
            System.loadLibrary("gstreamer_android");
            System.loadLibrary("tutorial-2");
            nativeClassInit();
        }

    }

As usual, the first bit that gets executed is the static initializer of the class:

    static {
        System.loadLibrary("gstreamer_android");
        System.loadLibrary("tutorial-2");
        nativeClassInit();
    }

As explained in the previous tutorial, the two native libraries are loaded and their `JNI_OnLoad()` methods are executed. Here, we also call the native method `nativeClassInit()`, previously declared with the `native` keyword in line 19. We will later see what its purpose is

In the `onCreate()` method GStreamer is initialized as in the previous tutorial with `GStreamer.init(this)`, and then the layout is inflated and listeners are setup for the two UI buttons:

    ImageButton play = (ImageButton) this.findViewById(R.id.button_play);
    play.setOnClickListener(new OnClickListener() {
        public void onClick(View v) {
            is_playing_desired = true;
            nativePlay();
        }
    });
    ImageButton pause = (ImageButton) this.findViewById(R.id.button_stop);
    pause.setOnClickListener(new OnClickListener() {
        public void onClick(View v) {
            is_playing_desired = false;
            nativePause();
        }
    });

Each button instructs the native code to set the pipeline to the desired state, and also remembers this state in the `is_playing_desired` variable. This is required so, when the application is restarted (for example, due to an orientation change), it can set the pipeline again to the desired state. This approach is easier and safer than tracking the actual pipeline state, because orientation changes can happen before the pipeline has moved to the desired state, for example.

    if (savedInstanceState != null) {
        is_playing_desired = savedInstanceState.getBoolean("playing");
        Log.i ("GStreamer", "Activity created. Saved state is playing:" + is_playing_desired);
    } else {
        is_playing_desired = false;
        Log.i ("GStreamer", "Activity created. There is no saved state, playing: false");
    }

Restore the previous playing state (if any) from `savedInstanceState`. We will first build the GStreamer pipeline (below) and only when the native code reports itself as initialized we will use `is_playing_desired`.

    nativeInit();

As will be shown in the C code, `nativeInit()` creates a dedicated thread, a GStreamer pipeline, a GLib main loop, and, right before calling `g_main_loop_run()` and going to sleep, it warns the Java code that the native code is initialized and ready to accept commands.

This finishes the `onCreate()` method and the Java initialization. The UI buttons are disabled, so nothing will happen until native code is ready and `onGStreamerInitialized()` is called:

    private void onGStreamerInitialized () {
        Log.i ("GStreamer", "Gst initialized. Restoring state, playing:" + is_playing_desired);

This is called by the native code when its main loop is finally running. We first retrieve the desired playing state from `is_playing_desired`, and then set that state:

    // Restore previous playing state
    if (is_playing_desired) {
        nativePlay();
    } else {
        nativePause();
    }

Here comes the first caveat, when re-enabling the UI buttons:

    // Re-enable buttons, now that GStreamer is initialized
    final Activity activity = this;
    runOnUiThread(new Runnable() {
        public void run() {
            activity.findViewById(R.id.button_play).setEnabled(true);
            activity.findViewById(R.id.button_stop).setEnabled(true);
        }
    });

This method is being called from the thread that the native code created to run its main loop, and is not allowed to issue UI-altering commands: Only the UI thread can do that. The solution is easy though: Android Activities have a handy runOnUiThread() method which lets bits of code to be executed from the correct thread. A Runnable instance has to be constructed and any parameter can be passed either by sub-classing Runnable and adding a dedicated constructor, or by using the `final` modifier, as shown in the above snippet.

The same problem exists when the native code wants to output a string in our TextView using the `setMessage()` method: it has to be done from the UI thread. The solution is the same:

    private void setMessage(final String message) {
        final TextView tv = (TextView) this.findViewById(R.id.textview_message);
        runOnUiThread (new Runnable() {
          public void run() {
          tv.setText(message);
        }
      });
    }

Finally, a few remaining bits:

    protected void onSaveInstanceState (Bundle outState) {
        Log.d ("GStreamer", "Saving state, playing:" + is_playing_desired);
        outState.putBoolean("playing", is_playing_desired);
    }

This method stores the currently desired playing state when Android is about to shut us down, so next time it restarts (after an orientation change, for example), it can restore the same state.

    protected void onDestroy() {
        nativeFinalize();
        super.onDestroy();
    }

And this is called before Android destroys our application. We call the `nativeFinalize()`method to exit the main loop, destroy its thread and all allocated resources.

This concludes the UI part of the tutorial.

### A pipeline on Android [C code]

jni/tutorial-2.c

    #include <string.h>
    #include <jni.h>
    #include <android/log.h>
    #include <gst/gst.h>
    #include <pthread.h>

    GST_DEBUG_CATEGORY_STATIC (debug_category);
    #define GST_CAT_DEFAULT debug_category

    /*
     * These macros provide a way to store the native pointer to CustomData, which might be 32 or 64 bits, into
     * a jlong, which is always 64 bits, without warnings.
     */
    #if GLIB_SIZEOF_VOID_P == 8
    # define GET_CUSTOM_DATA(env, thiz, fieldID) (CustomData *)(*env)->GetLongField (env, thiz, fieldID)
    # define SET_CUSTOM_DATA(env, thiz, fieldID, data) (*env)->SetLongField (env, thiz, fieldID, (jlong)data)
    #else
    # define GET_CUSTOM_DATA(env, thiz, fieldID) (CustomData *)(jint)(*env)->GetLongField (env, thiz, fieldID)
    # define SET_CUSTOM_DATA(env, thiz, fieldID, data) (*env)->SetLongField (env, thiz, fieldID, (jlong)(jint)data)
    #endif

    /* Structure to contain all our information, so we can pass it to callbacks */
    typedef struct _CustomData {
      jobject app;           /* Application instance, used to call its methods. A global reference is kept. */
      GstElement *pipeline;  /* The running pipeline */
      GMainContext *context; /* GLib context used to run the main loop */
      GMainLoop *main_loop;  /* GLib main loop */
      gboolean initialized;  /* To avoid informing the UI multiple times about the initialization */
    } CustomData;

    /* These global variables cache values which are not changing during execution */
    static pthread_t gst_app_thread;
    static pthread_key_t current_jni_env;
    static JavaVM *java_vm;
    static jfieldID custom_data_field_id;
    static jmethodID set_message_method_id;
    static jmethodID on_gstreamer_initialized_method_id;

    /*
     * Private methods
     */

    /* Register this thread with the VM */
    static JNIEnv *attach_current_thread (void) {
      JNIEnv *env;
      JavaVMAttachArgs args;

      GST_DEBUG ("Attaching thread %p", g_thread_self ());
      args.version = JNI_VERSION_1_4;
      args.name = NULL;
      args.group = NULL;

      if ((*java_vm)->AttachCurrentThread (java_vm, &env, &args) < 0) {
        GST_ERROR ("Failed to attach current thread");
        return NULL;
      }

      return env;
    }

    /* Unregister this thread from the VM */
    static void detach_current_thread (void *env) {
      GST_DEBUG ("Detaching thread %p", g_thread_self ());
      (*java_vm)->DetachCurrentThread (java_vm);
    }

    /* Retrieve the JNI environment for this thread */
    static JNIEnv *get_jni_env (void) {
      JNIEnv *env;

      if ((env = pthread_getspecific (current_jni_env)) == NULL) {
        env = attach_current_thread ();
        pthread_setspecific (current_jni_env, env);
      }

      return env;
    }

    /* Change the content of the UI's TextView */
    static void set_ui_message (const gchar *message, CustomData *data) {
      JNIEnv *env = get_jni_env ();
      GST_DEBUG ("Setting message to: %s", message);
      jstring jmessage = (*env)->NewStringUTF(env, message);
      (*env)->CallVoidMethod (env, data->app, set_message_method_id, jmessage);
      if ((*env)->ExceptionCheck (env)) {
        GST_ERROR ("Failed to call Java method");
        (*env)->ExceptionClear (env);
      }
      (*env)->DeleteLocalRef (env, jmessage);
    }

    /* Retrieve errors from the bus and show them on the UI */
    static void error_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
      GError *err;
      gchar *debug_info;
      gchar *message_string;

      gst_message_parse_error (msg, &err, &debug_info);
      message_string = g_strdup_printf ("Error received from element %s: %s", GST_OBJECT_NAME (msg->src), err->message);
      g_clear_error (&err);
      g_free (debug_info);
      set_ui_message (message_string, data);
      g_free (message_string);
      gst_element_set_state (data->pipeline, GST_STATE_NULL);
    }

    /* Notify UI about pipeline state changes */
    static void state_changed_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
      GstState old_state, new_state, pending_state;
      gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
      /* Only pay attention to messages coming from the pipeline, not its children */
      if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data->pipeline)) {
        gchar *message = g_strdup_printf("State changed to %s", gst_element_state_get_name(new_state));
        set_ui_message(message, data);
        g_free (message);
      }
    }

    /* Check if all conditions are met to report GStreamer as initialized.
     * These conditions will change depending on the application */
    static void check_initialization_complete (CustomData *data) {
      JNIEnv *env = get_jni_env ();
      if (!data->initialized && data->main_loop) {
        GST_DEBUG ("Initialization complete, notifying application. main_loop:%p", data->main_loop);
        (*env)->CallVoidMethod (env, data->app, on_gstreamer_initialized_method_id);
        if ((*env)->ExceptionCheck (env)) {
          GST_ERROR ("Failed to call Java method");
          (*env)->ExceptionClear (env);
        }
        data->initialized = TRUE;
      }
    }

    /* Main method for the native code. This is executed on its own thread. */
    static void *app_function (void *userdata) {
      JavaVMAttachArgs args;
      GstBus *bus;
      CustomData *data = (CustomData *)userdata;
      GSource *bus_source;
      GError *error = NULL;

      GST_DEBUG ("Creating pipeline in CustomData at %p", data);

      /* Create our own GLib Main Context and make it the default one */
      data->context = g_main_context_new ();
      g_main_context_push_thread_default(data->context);

      /* Build pipeline */
      data->pipeline = gst_parse_launch("audiotestsrc ! audioconvert ! audioresample ! autoaudiosink", &error);
      if (error) {
        gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message);
        g_clear_error (&error);
        set_ui_message(message, data);
        g_free (message);
        return NULL;
      }

      /* Instruct the bus to emit signals for each received message, and connect to the interesting signals */
      bus = gst_element_get_bus (data->pipeline);
      bus_source = gst_bus_create_watch (bus);
      g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);
      g_source_attach (bus_source, data->context);
      g_source_unref (bus_source);
      g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, data);
      g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, data);
      gst_object_unref (bus);

      /* Create a GLib Main Loop and set it to run */
      GST_DEBUG ("Entering main loop... (CustomData:%p)", data);
      data->main_loop = g_main_loop_new (data->context, FALSE);
      check_initialization_complete (data);
      g_main_loop_run (data->main_loop);
      GST_DEBUG ("Exited main loop");
      g_main_loop_unref (data->main_loop);
      data->main_loop = NULL;

      /* Free resources */
      g_main_context_pop_thread_default(data->context);
      g_main_context_unref (data->context);
      gst_element_set_state (data->pipeline, GST_STATE_NULL);
      gst_object_unref (data->pipeline);

      return NULL;
    }

    /*
     * Java Bindings
     */

    /* Instruct the native code to create its internal data structure, pipeline and thread */
    static void gst_native_init (JNIEnv* env, jobject thiz) {
      CustomData *data = g_new0 (CustomData, 1);
      SET_CUSTOM_DATA (env, thiz, custom_data_field_id, data);
      GST_DEBUG_CATEGORY_INIT (debug_category, "tutorial-2", 0, "Android tutorial 2");
      gst_debug_set_threshold_for_name("tutorial-2", GST_LEVEL_DEBUG);
      GST_DEBUG ("Created CustomData at %p", data);
      data->app = (*env)->NewGlobalRef (env, thiz);
      GST_DEBUG ("Created GlobalRef for app object at %p", data->app);
      pthread_create (&gst_app_thread, NULL, &app_function, data);
    }

    /* Quit the main loop, remove the native thread and free resources */
    static void gst_native_finalize (JNIEnv* env, jobject thiz) {
      CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
      if (!data) return;
      GST_DEBUG ("Quitting main loop...");
      g_main_loop_quit (data->main_loop);
      GST_DEBUG ("Waiting for thread to finish...");
      pthread_join (gst_app_thread, NULL);
      GST_DEBUG ("Deleting GlobalRef for app object at %p", data->app);
      (*env)->DeleteGlobalRef (env, data->app);
      GST_DEBUG ("Freeing CustomData at %p", data);
      g_free (data);
      SET_CUSTOM_DATA (env, thiz, custom_data_field_id, NULL);
      GST_DEBUG ("Done finalizing");
    }

    /* Set pipeline to PLAYING state */
    static void gst_native_play (JNIEnv* env, jobject thiz) {
      CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
      if (!data) return;
      GST_DEBUG ("Setting state to PLAYING");
      gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
    }

    /* Set pipeline to PAUSED state */
    static void gst_native_pause (JNIEnv* env, jobject thiz) {
      CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
      if (!data) return;
      GST_DEBUG ("Setting state to PAUSED");
      gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
    }

    /* Static class initializer: retrieve method and field IDs */
    static jboolean gst_native_class_init (JNIEnv* env, jclass klass) {
      custom_data_field_id = (*env)->GetFieldID (env, klass, "native_custom_data", "J");
      set_message_method_id = (*env)->GetMethodID (env, klass, "setMessage", "(Ljava/lang/String;)V");
      on_gstreamer_initialized_method_id = (*env)->GetMethodID (env, klass, "onGStreamerInitialized", "()V");

      if (!custom_data_field_id || !set_message_method_id || !on_gstreamer_initialized_method_id) {
        /* We emit this message through the Android log instead of the GStreamer log because the later
         * has not been initialized yet.
         */
        __android_log_print (ANDROID_LOG_ERROR, "tutorial-2", "The calling class does not implement all necessary interface methods");
        return JNI_FALSE;
      }
      return JNI_TRUE;
    }

    /* List of implemented native methods */
    static JNINativeMethod native_methods[] = {
      { "nativeInit", "()V", (void *) gst_native_init},
      { "nativeFinalize", "()V", (void *) gst_native_finalize},
      { "nativePlay", "()V", (void *) gst_native_play},
      { "nativePause", "()V", (void *) gst_native_pause},
      { "nativeClassInit", "()Z", (void *) gst_native_class_init}
    };

    /* Library initializer */
    jint JNI_OnLoad(JavaVM *vm, void *reserved) {
      JNIEnv *env = NULL;

      java_vm = vm;

      if ((*vm)->GetEnv(vm, (void**) &env, JNI_VERSION_1_4) != JNI_OK) {
        __android_log_print (ANDROID_LOG_ERROR, "tutorial-2", "Could not retrieve JNIEnv");
        return 0;
      }
      jclass klass = (*env)->FindClass (env, "org/freedesktop/gstreamer/tutorials/tutorial_2/Tutorial2");
      (*env)->RegisterNatives (env, klass, native_methods, G_N_ELEMENTS(native_methods));

      pthread_key_create (&current_jni_env, detach_current_thread);

      return JNI_VERSION_1_4;
    }

Let’s start with the `CustomData` structure. We have seen it in most of the basic tutorials, and it is used to hold all our information in one place, so we can easily pass it around to callbacks:

    /* Structure to contain all our information, so we can pass it to callbacks */
    typedef struct _CustomData {
      jobject app;           /* Application instance, used to call its methods. A global reference is kept. */
      GstElement *pipeline;  /* The running pipeline */
      GMainContext *context; /* GLib context used to run the main loop */
      GMainLoop *main_loop;  /* GLib main loop */
      gboolean initialized;  /* To avoid informing the UI multiple times about the initialization */
    } CustomData;

We will see the meaning of each member as we go. What is interesting now is that `CustomData` belongs to the application, so a pointer is kept in the Tutorial2 Java class in the `private long native_custom_data` attribute. Java only holds this pointer for us; it is completely handled in C code.

From C, this pointer can be set and retrieved with the SetLongField() and GetLongField() JNI functions, but two convenience macros have been defined, `SET_CUSTOM_DATA` and `GET_CUSTOM_DATA`. These macros are handy because the `long` type used in Java is always 64 bits wide, but the pointer used in C can be either 32 or 64 bits wide. The macros take care of the conversion without warnings.

    /* Library initializer */
    jint JNI_OnLoad(JavaVM *vm, void *reserved) {
      JNIEnv *env = NULL;

      java_vm = vm;

      if ((*vm)->GetEnv(vm, (void**) &env, JNI_VERSION_1_4) != JNI_OK) {
        __android_log_print (ANDROID_LOG_ERROR, "tutorial-2", "Could not retrieve JNIEnv");
        return 0;
      }
      jclass klass = (*env)->FindClass (env, "org/freedesktop/gstreamer/tutorials/tutorial_2/Tutorial2");
      (*env)->RegisterNatives (env, klass, native_methods, G_N_ELEMENTS(native_methods));

      pthread_key_create (&current_jni_env, detach_current_thread);

      return JNI_VERSION_1_4;
    }

The `JNI_OnLoad` function is almost the same as the previous tutorial. It registers the list of native methods (which is longer in this tutorial). It also uses pthread_key_create() to be able to store per-thread information, which is crucial to properly manage the JNI Environment, as shown later.

    /* Static class initializer: retrieve method and field IDs */
    static jboolean gst_native_class_init (JNIEnv* env, jclass klass) {
      custom_data_field_id = (*env)->GetFieldID (env, klass, "native_custom_data", "J");
      set_message_method_id = (*env)->GetMethodID (env, klass, "setMessage", "(Ljava/lang/String;)V");
      on_gstreamer_initialized_method_id = (*env)->GetMethodID (env, klass, "onGStreamerInitialized", "()V");

      if (!custom_data_field_id || !set_message_method_id || !on_gstreamer_initialized_method_id) {
        /* We emit this message through the Android log instead of the GStreamer log because the later
         * has not been initialized yet.
         */
        __android_log_print (ANDROID_LOG_ERROR, "tutorial-2", "The calling class does not implement all necessary interface methods");
        return JNI_FALSE;
      }
      return JNI_TRUE;
    }

This method is called from the static initializer of the Java class, which is passed as a parameter (since this is called from a static method, it receives a class object instead of an instance object). In order for C code to be able to call a Java method, it needs to know the method’s MethodID. This ID is obtained from the method’s name and signature and can be cached. The purpose of the `gst_native_class_init()` function is to obtain the IDs of all the methods and fields that the C code will need. If some ID cannot be retrieved, the calling Java class does not offer the expected interface and execution should halt (which is not currently done for simplicity).

Let’s review now the first native method which can be directly called from Java:

#### `gst_native_init()` (`nativeInit()` from Java)

This method is called at the end of Java's `onCreate()`.

    static void gst_native_init (JNIEnv* env, jobject thiz) {
      CustomData *data = g_new0 (CustomData, 1);
      SET_CUSTOM_DATA (env, thiz, custom_data_field_id, data);

It first allocates memory for the `CustomData` structure and passes the pointer to the Java class with `SET_CUSTOM_DATA`, so it is remembered.

    data->app = (*env)->NewGlobalRef (env, thiz);

A pointer to the application class (the `Tutorial2` class) is also kept in `CustomData` (a Global Reference is used) so its methods can be called later.

    pthread_create (&gst_app_thread, NULL, &app_function, data);

Finally, a thread is created and it starts running the `app_function()` method.

#### `app_function()`

    /* Main method for the native code. This is executed on its own thread. */
    static void *app_function (void *userdata) {
      JavaVMAttachArgs args;
      GstBus *bus;
      CustomData *data = (CustomData *)userdata;
      GSource *bus_source;
      GError *error = NULL;

      GST_DEBUG ("Creating pipeline in CustomData at %p", data);

      /* Create our own GLib Main Context and make it the default one */
      data->context = g_main_context_new ();
      g_main_context_push_thread_default(data->context);

It first creates a GLib context so all `GSource` are kept in the same place. This also helps cleaning after GSources created by other libraries which might not have been properly disposed of. A new context is created with `g_main_context_new()` and then it is made the default one for the thread with `g_main_context_push_thread_default()`.

    data->pipeline = gst_parse_launch("audiotestsrc ! audioconvert ! audioresample ! autoaudiosink", &error);
    if (error) {
      gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message);
      g_clear_error (&error);
      set_ui_message(message, data);
      g_free (message);
      return NULL;
    }

It then creates a pipeline the easy way, with `gst-parse-launch()`. In this case, it is simply an `audiotestsrc` (which produces a continuous tone) and an `autoaudiosink`, with accompanying adapter elements.

    bus = gst_element_get_bus (data->pipeline);
    bus_source = gst_bus_create_watch (bus);
    g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);
    g_source_attach (bus_source, data->context);
    g_source_unref (bus_source);
    g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, data);
    g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, data);
    gst_object_unref (bus);

These lines create a bus signal watch and connect to some interesting signals, just like we have been doing in the basic tutorials. The creation of the watch is done step by step instead of using `gst_bus_add_signal_watch()` to exemplify how to use a custom GLib context.

    GST_DEBUG ("Entering main loop... (CustomData:%p)", data);
    data->main_loop = g_main_loop_new (data->context, FALSE);
    check_initialization_complete (data);
    g_main_loop_run (data->main_loop);
    GST_DEBUG ("Exited main loop");
    g_main_loop_unref (data->main_loop);
    data->main_loop = NULL;

Finally, the main loop is created and set to run. When it exits (because somebody else calls `g_main_loop_quit()`) the main loop is disposed of. Before entering the main loop, though, `check_initialization_complete()` is called. This method checks if all conditions are met to consider the native code “ready” to accept commands. Since having a running main loop is one of the conditions, `check_initialization_complete()` is called here. This method is reviewed below.

Once the main loop has quit, all resources are freed in lines 178 to 181.

#### `check_initialization_complete()`

    static void check_initialization_complete (CustomData *data) {
      JNIEnv *env = get_jni_env ();
      if (!data->initialized && data->main_loop) {
        GST_DEBUG ("Initialization complete, notifying application. main_loop:%p", data->main_loop);
        (*env)->CallVoidMethod (env, data->app, on_gstreamer_initialized_method_id);
        if ((*env)->ExceptionCheck (env)) {
          GST_ERROR ("Failed to call Java method");
          (*env)->ExceptionClear (env);
        }
        data->initialized = TRUE;
      }
    }

This method does not do much in this tutorial, but it will also be used in the next ones, with progressively more complex functionality. Its purpose is to check if the native code is ready to accept commands, and, if so, notify the UI code.

In tutorial 2, the only conditions are 1) the code is not already initialized and 2) the main loop is running. If these two are met, the Java `onGStreamerInitialized()` method is called via the CallVoidMethod() JNI call.

Here comes a tricky bit. JNI calls require a JNI Environment, which is different for every thread. C methods called from Java receive a `JNIEnv` pointer as a parameter, but this is not the situation with `check_initialization_complete()`. Here, we are in a thread which has never been called from Java, so we have no `JNIEnv`. We need to use the `JavaVM` pointer (passed to us in the `JNI_OnLoad()` method, and shared among all threads) to attach this thread to the Java Virtual Machine and obtain a `JNIEnv`. This `JNIEnv` is stored in the Thread-Local Storage (TLS) using the pthread key we created in `JNI_OnLoad()`, so we do not need to attach the thread anymore.

This behavior is implemented in the `get_jni_env()` method, used for example in `check_initialization_complete()` as we have just seen. Let’s see how it works, step by step:

#### `get_jni_env()`

    static JNIEnv *get_jni_env (void) {
      JNIEnv *env;
      if ((env = pthread_getspecific (current_jni_env)) == NULL) {
        env = attach_current_thread ();
        pthread_setspecific (current_jni_env, env);
      }
      return env;
    }

It first retrieves the current `JNIEnv` from the TLS using pthread_getspecific() and the key we obtained from pthread_key_create(). If it returns NULL, we never attached this thread, so we do now with `attach_current_thread()` and then store the new `JNIEnv` into the TLS with pthread_setspecific().

#### `attach_current_thread()`

This method is simply a convenience wrapper around AttachCurrentThread() to deal with its parameters.

#### `detach_current_thread()`

This method is called by the pthreads library when a TLS key is deleted, meaning that the thread is about to be destroyed. We simply detach the thread from the JavaVM with DetachCurrentThread().

Let's now review the rest of the native methods accessible from Java:

#### `gst_native_finalize()` (`nativeFinalize()` from Java)

    static void gst_native_finalize (JNIEnv* env, jobject thiz) {
      CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
      if (!data) return;
      GST_DEBUG ("Quitting main loop...");
      g_main_loop_quit (data->main_loop);
      GST_DEBUG ("Waiting for thread to finish...");
      pthread_join (gst_app_thread, NULL);
      GST_DEBUG ("Deleting GlobalRef for app object at %p", data->app);
      (*env)->DeleteGlobalRef (env, data->app);
      GST_DEBUG ("Freeing CustomData at %p", data);
      g_free (data);
      SET_CUSTOM_DATA (env, thiz, custom_data_field_id, NULL);
      GST_DEBUG ("Done finalizing");
    }

This method is called from Java in `onDestroy()`, when the activity is about to be destroyed. Here, we:

- Instruct the GLib main loop to quit with `g_main_loop_quit()`. This call returns immediately, and the main loop will terminate at its earliest convenience.
- Wait for the thread to finish with pthread_join(). This call blocks until the `app_function()` method returns, meaning that the main loop has exited, and the thread has been destroyed.
- Dispose of the global reference we kept for the Java application class (`Tutorial2`) in `CustomData`.
- Free `CustomData` and set the Java pointer inside the `Tutorial2` class to NULL with `SET_CUSTOM_DATA()`.

#### `gst_native_play` and `gst_native_pause()` (`nativePlay` and `nativePause()` from Java)

These two simple methods retrieve `CustomData` from the passed-in object with `GET_CUSTOM_DATA()` and set the pipeline found inside `CustomData` to the desired state, returning immediately.

Finally, let’s see how the GStreamer callbacks are handled:

#### `error_cb` and `state_changed_cb`

This tutorial does not do much in these callbacks. They simply parse the error or state changed message and display a message in the UI using the `set_ui_message()` method:

#### `set_ui_message()`

    static void set_ui_message (const gchar *message, CustomData *data) {
      JNIEnv *env = get_jni_env ();
      GST_DEBUG ("Setting message to: %s", message);
      jstring jmessage = (*env)->NewStringUTF(env, message);
      (*env)->CallVoidMethod (env, data->app, set_message_method_id, jmessage);
      if ((*env)->ExceptionCheck (env)) {
        GST_ERROR ("Failed to call Java method");
        (*env)->ExceptionClear (env);
      }
      (*env)->DeleteLocalRef (env, jmessage);
    }

This is the other method (besides `check_initialization_complete()`) that needs to call a Java function from a thread which never received an `JNIEnv` pointer directly. Notice how all the complexities of attaching the thread to the JavaVM and storing the JNI environment in the TLS are hidden in the simple call to `get_jni_env()`.

The desired message (received in ASCII, or modified UTF8), is converted to UTF16 as required by Java using the NewStringUTF() JNI call.

The `setMessage()` Java method is called via the JNI CallVoidMethod() using the global reference to the class we are keeping in `CustomData` (`data->app`) and the `set_message_method_id` we cached in `gst_native_class_init()`.

We check for exceptions with the JNI ExceptionCheck() method and free the UTF16 message with DeleteLocalRef().

### A pipeline on Android [Android.mk]

jni/Android.mk

    LOCAL_PATH := $(call my-dir)

    include $(CLEAR_VARS)

    LOCAL_MODULE    := tutorial-2
    LOCAL_SRC_FILES := tutorial-2.c
    LOCAL_SHARED_LIBRARIES := gstreamer_android
    LOCAL_LDLIBS := -llog
    include $(BUILD_SHARED_LIBRARY)

    ifndef GSTREAMER_ROOT
    ifndef GSTREAMER_ROOT_ANDROID
    $(error GSTREAMER_ROOT_ANDROID is not defined!)
    endif
    GSTREAMER_ROOT        := $(GSTREAMER_ROOT_ANDROID)
    endif
    GSTREAMER_NDK_BUILD_PATH  := $(GSTREAMER_ROOT)/share/gst-android/ndk-build/
    include $(GSTREAMER_NDK_BUILD_PATH)/plugins.mk
    GSTREAMER_PLUGINS         := $(GSTREAMER_PLUGINS_CORE) $(GSTREAMER_PLUGINS_SYS)
    include $(GSTREAMER_NDK_BUILD_PATH)/gstreamer-1.0.mk

Notice how the required `GSTREAMER_PLUGINS` are now `$(GSTREAMER_PLUGINS_CORE)` (For the test source and converter elements) and `$(GSTREAMER_PLUGINS_SYS)` (for the audio sink).

And this is it! This has been a rather long tutorial, but we covered a lot of territory. Building on top of this one, the following ones are shorter and focus only on the new topics.

### Conclusion

This tutorial has shown:

- How to manage multiple threads from C code and have them interact with java.
- How to access Java code from any C thread using AttachCurrentThread().
- How to allocate a CustomData structure from C and have Java host it, so it is available to all threads.

Most of the methods introduced in this tutorial, like `get_jni_env()`, `check_initialization_complete()`, `app_function()` and the API methods `gst_native_init()`, `gst_native_finalize()` and `gst_native_class_init()` will continue to be used in the following tutorials with minimal modifications, so better get used to them!

As usual, it has been a pleasure having you here, and see you soon!

The results of the search are