Skip to main content

Twilio Video Adobe Native Extension - Building Android Library - Part 2

Now, We have build the interface for our extension. If you missed that part, please take a look here. Follow below steps to build android library.

Create android project

Open Android Studio and create an android project. Please give a package name(Here I am using same as wrapper package name) and give the same minimum sdk Twilio video will support. Don't create activity now because we will be creating later.

Add Adobe extension library

Now copy FlashRuntimeExtensions.jar file from lib/android directory in Flex SDK directory to app/libs directory in your android project.

Later, you need to add this jar library to your project. Go to File->Project Structure, Under Modules section, select app. Here you can see your project settings.

Under dependency tab, add a new jar dependency and select the jar file we have copied. When you add the jar file, you can see our jar file added in app gradle file.

Add Twilio video

Now add latest twilio video(currently available version is 2.1.0). You can find how to add twilio video from https://bintray.com/twilio/releases/video-android.

Create library extension initializer

Now create a java class named VideoExtension.java under your packaged folder. This class should implement FREExtension. So our wrapper will be contacting this class when extension is initiatlised.

Copy below code to our new file
package ijasnahamed.twilio.video;



import com.adobe.fre.FREContext;

import com.adobe.fre.FREExtension;



public class VideoExtension implements FREExtension {

    public static FREContext extensionContext = null;



    @Override

    public void initialize() {



    }



    @Override

    public FREContext createContext(String s) {

        return new VideoContext();

    }



    @Override

    public void dispose() {

        extensionContext = null;

    }

}

Static variable extensionContext will be used to send data from our video activity to wrapper, so it will send to the AIR app. CreateContext method will return the context of the libary. Currently we haven't implemented the context. So it will give an error for now.

Create extension context

Now create another java class named VideoContext.java which will inherit FREContext class. This class will be the context for our library.

package ijasnahamed.twilio.video;



import com.adobe.fre.FREContext;

import com.adobe.fre.FREFunction;



import java.util.HashMap;

import java.util.Map;



import ijasnahamed.twilio.video.functions.VideoFunction;



public class VideoContext extends FREContext {

    @Override

    public Map<String, FREFunction> getFunctions() {

        Map<String, FREFunction> functions = new HashMap<String, FREFunction>();



        functions.put("start", new VideoFunction());



        return functions;

    }



    @Override

    public void dispose() {



    }

}

getFunctions method will return extension method implementers. Currently, we have start as a method in extension, so we will be returning a map with start implementer.

Implement video start function handler

Create a package name functions and add a new java class with name VideoFunction. This class should implement FREFunction. Wrapper will invoke this class call method when our AIR app calls start method.

package ijasnahamed.twilio.video.functions;



import android.app.Activity;

import android.content.Context;

import android.content.Intent;



import com.adobe.fre.FREContext;

import com.adobe.fre.FREFunction;

import com.adobe.fre.FREInvalidObjectException;

import com.adobe.fre.FREObject;

import com.adobe.fre.FRETypeMismatchException;

import com.adobe.fre.FREWrongThreadException;



import ijasnahamed.twilio.video.VideoExtension;

import ijasnahamed.twilio.video.utils.CommonStuff;

import ijasnahamed.twilio.video.views.VideoActivity;



public class VideoFunction implements FREFunction {

    @Override

    public FREObject call(FREContext freContext, FREObject[] args) {

        VideoExtension.extensionContext = freContext;



        Activity activity = freContext.getActivity();

        Context context = activity.getApplicationContext();



        String callDetails = "";

        try {

            callDetails = args[0].getAsString();

        } catch (FRETypeMismatchException e) {



        } catch (FREInvalidObjectException e) {



        } catch (FREWrongThreadException e) {



        }



        Intent intent = new Intent(context, VideoActivity.class);

        intent.putExtra("layoutId"

                , freContext.getResourceId("layout.ijasnahamed_twilio_video_activity"));

        intent.putExtra("hangupBtnId", freContext.getResourceId("id.DisconnectBtn"));

        intent.putExtra("remoteViewId", freContext.getResourceId("id.RemoteView"));

        intent.putExtra("localViewId", freContext.getResourceId("id.LocalView"));

        intent.putExtra("callDetails", callDetails);



        activity.startActivity(intent);



        return null;

    }

}

This file will give error for VideoActivity because we haven't created our activity yet.

Create video activity dependencies

Create another package inside root package name utils. Under view, create a class named Pair and copy below code to that.

package ijasnahamed.twilio.video.utils;



import com.twilio.video.CameraCapturer;



public class Pair {

    public CameraCapturer.CameraSource cameraSource;

    public String cameraId;



    public Pair(CameraCapturer.CameraSource cameraSource, String cameraId) {

        this.cameraSource = cameraSource;

        this.cameraId = cameraId;

    }

}

Also, create another class named CameraCapturerCompat in utils package.

package ijasnahamed.twilio.video.utils;

import android.content.Context;



import com.twilio.video.Camera2Capturer;

import com.twilio.video.CameraCapturer;

import com.twilio.video.VideoCapturer;



import org.webrtc.Camera2Enumerator;



public class CameraCapturerCompat {


    private CameraCapturer camera1Capturer;

    private Camera2Capturer camera2Capturer;

    private Pair frontCameraPair;

    private Pair backCameraPair;

    private final Camera2Capturer.Listener camera2Listener = new Camera2Capturer.Listener() {

        @Override

        public void onFirstFrameAvailable() {
        }



        @Override

        public void onCameraSwitched(String newCameraId) {
        }



        @Override

        public void onError(Camera2Capturer.Exception camera2CapturerException) {
        }

    };



    public CameraCapturerCompat(Context context,

                                CameraCapturer.CameraSource cameraSource) {

        if (Camera2Capturer.isSupported(context)) {

            setCameraPairs(context);

            camera2Capturer = new Camera2Capturer(context,

                    getCameraId(cameraSource),

                    camera2Listener);

        } else {

            camera1Capturer = new CameraCapturer(context, cameraSource);

        }

    }



    public CameraCapturer.CameraSource getCameraSource() {

        if (usingCamera1()) {

            return camera1Capturer.getCameraSource();

        } else {

            return getCameraSource(camera2Capturer.getCameraId());

        }

    }



    public void switchCamera() {

        if (usingCamera1()) {

            camera1Capturer.switchCamera();

        } else {

            CameraCapturer.CameraSource cameraSource = getCameraSource(camera2Capturer

                    .getCameraId());



            if (cameraSource == CameraCapturer.CameraSource.FRONT_CAMERA) {

                camera2Capturer.switchCamera(backCameraPair.cameraId);

            } else {

                camera2Capturer.switchCamera(frontCameraPair.cameraId);

            }

        }

    }



    public VideoCapturer getVideoCapturer() {

        if (usingCamera1()) {

            return camera1Capturer;

        } else {

            return camera2Capturer;

        }

    }



    private boolean usingCamera1() {

        return camera1Capturer != null;

    }



    private void setCameraPairs(Context context) {

        Camera2Enumerator camera2Enumerator = new Camera2Enumerator(context);

        for (String cameraId : camera2Enumerator.getDeviceNames()) {

            if (camera2Enumerator.isFrontFacing(cameraId)) {

                frontCameraPair = new Pair(CameraCapturer.CameraSource.FRONT_CAMERA, cameraId);

            }

            if (camera2Enumerator.isBackFacing(cameraId)) {

                backCameraPair = new Pair(CameraCapturer.CameraSource.BACK_CAMERA, cameraId);

            }

        }

    }



    private String getCameraId(CameraCapturer.CameraSource cameraSource) {

        if (frontCameraPair.cameraSource == cameraSource) {

            return frontCameraPair.cameraId;

        } else {

            return backCameraPair.cameraId;

        }

    }



    private CameraCapturer.CameraSource getCameraSource(String cameraId) {

        if (frontCameraPair.cameraId.equals(cameraId)) {

            return frontCameraPair.cameraSource;

        } else {

            return backCameraPair.cameraSource;

        }

    }

}

CameraCapturerCompat class will hold the data of our phone cameras.

Create video activity with layout

Create another package named views under app package. Create an empty activity named VideoActivity with layout a layout file. We are starting this activity for video call from Step 6.

VideoActivity.java


package ijasnahamed.twilio.video.views;



import android.Manifest;

import android.app.ActionBar;

import android.app.Activity;

import android.app.AlertDialog;

import android.content.BroadcastReceiver;

import android.content.Context;

import android.content.DialogInterface;

import android.content.Intent;

import android.content.IntentFilter;

import android.content.pm.PackageManager;

import android.media.AudioAttributes;

import android.media.AudioDeviceInfo;

import android.media.AudioFocusRequest;

import android.media.AudioManager;

import android.net.Uri;

import android.os.Build;

import android.os.Bundle;

import android.provider.Settings;

import android.support.annotation.NonNull;

import android.view.View;

import android.widget.Button;

import android.widget.TextView;

import android.widget.Toast;



import com.twilio.video.AudioCodec;

import com.twilio.video.CameraCapturer;

import com.twilio.video.ConnectOptions;

import com.twilio.video.EncodingParameters;

import com.twilio.video.G722Codec;

import com.twilio.video.H264Codec;

import com.twilio.video.IsacCodec;

import com.twilio.video.LocalAudioTrack;

import com.twilio.video.LocalParticipant;

import com.twilio.video.LocalVideoTrack;

import com.twilio.video.OpusCodec;

import com.twilio.video.PcmaCodec;

import com.twilio.video.PcmuCodec;

import com.twilio.video.RemoteAudioTrack;

import com.twilio.video.RemoteAudioTrackPublication;

import com.twilio.video.RemoteDataTrack;

import com.twilio.video.RemoteDataTrackPublication;

import com.twilio.video.RemoteParticipant;

import com.twilio.video.RemoteVideoTrack;

import com.twilio.video.RemoteVideoTrackPublication;

import com.twilio.video.Room;

import com.twilio.video.RoomState;

import com.twilio.video.TwilioException;

import com.twilio.video.Video;

import com.twilio.video.VideoCodec;

import com.twilio.video.VideoTrack;

import com.twilio.video.VideoView;

import com.twilio.video.Vp8Codec;

import com.twilio.video.Vp9Codec;



import org.json.JSONException;

import org.json.JSONObject;



import java.util.Collections;



import ijasnahamed.twilio.video.VideoExtension;

import ijasnahamed.twilio.video.utils.CameraCapturerCompat;



public class VideoActivity extends Activity {

    private static final int CAMERA_MIC_PERMISSION_REQUEST_CODE = 1;



    private static final String LOCAL_AUDIO_TRACK_NAME = "mic";

    private static final String LOCAL_VIDEO_TRACK_NAME = "camera";



    private static final String EVENT_CALL_CONNECTED = "callConnected";

    private static final String EVENT_CALL_DISCONNECTED = "callDisconnected";

    private static final String EVENT_CALL_FAILED = "callFailed";

    private static final String EVENT_PARTICIPANT_CONNECTED = "participantConencted";

    private static final String EVENT_PARTICIPANT_DISCONNECTED = "participantDisconnected";

    private static final String EVENT_PARTICIPANT_AUDIO_TRACK_ADDED = "participantAudioTrackAdded";

    private static final String EVENT_PARTICIPANT_AUDIO_TRACK_REMOVED = "participantAudioTrackRemoved";

    private static final String EVENT_PARTICIPANT_VIDEO_TACK_ADDED = "participantVideoTrackAdded";

    private static final String EVENT_PARTICIPANT_VIDEO_TRACK_REMOVED = "participantVIdeoTrackRemoved";



    private static final String DEFAULT_AUDIO_CODEC = OpusCodec.NAME;

    private static final String DEFAULT_VIDEO_CODEC = Vp8Codec.NAME;

    private static final int DEFAULT_MAX_SENDER_AUDIO_BITRATE = 0;

    private static final int DEFAULT_MAX_SENDER_VIDEO_BITRATE = 0;



    private String accessToken, roomName;



    private Room room;

    private LocalParticipant localParticipant;



    private AudioCodec audioCodec;

    private VideoCodec videoCodec;



    private EncodingParameters encodingParameters;



    private VideoView primaryVideoView;

    private VideoView thumbnailVideoView;

    private Button hangupBtn;



    private CameraCapturerCompat cameraCapturerCompat;

    private LocalAudioTrack localAudioTrack;

    private LocalVideoTrack localVideoTrack;

    private AudioManager audioManager;

    private AlertDialog alertDialog;





    private String remoteParticipantIdentity = "";

    private boolean disconnectedFromOnDestroy;

    private int previousAudioMode;

    private boolean previousMicrophoneMute;



    @Override

    protected void onCreate(Bundle savedInstanceState) {

        super.onCreate(savedInstanceState);



        Intent currentIntent = getIntent();

        int layoutId = currentIntent.getIntExtra("layoutId", -1);

        if(layoutId == -1) {

            onBackPressed();

            return;

        }

        setContentView(layoutId);



        ActionBar actionBar = getActionBar();

        if(actionBar != null) {

            actionBar.hide();

        }



        int hangupBtnId = currentIntent.getIntExtra("hangupBtnId", -1);

        hangupBtn = (Button) findViewById(hangupBtnId);



        int remoteViewId = currentIntent.getIntExtra("remoteViewId", -1);

        int localViewId = currentIntent.getIntExtra("localViewId", -1);



        primaryVideoView = (VideoView) findViewById(remoteViewId);

        thumbnailVideoView = (VideoView) findViewById(localViewId);



        JSONObject callDetails;

        try {

            callDetails = new JSONObject(currentIntent.getStringExtra("callDetails"));

        } catch (JSONException e) {

            callDetails = new JSONObject();

        }



        accessToken = callDetails.optString("token");

        roomName = callDetails.optString("room");



        audioCodec = getAudioCodecPreference(DEFAULT_AUDIO_CODEC);

        videoCodec = getVideoCodecPreference(DEFAULT_VIDEO_CODEC);



        setVolumeControlStream(AudioManager.STREAM_VOICE_CALL);



        audioManager = (AudioManager)getSystemService(Context.AUDIO_SERVICE);



        IntentFilter intentFilter = new IntentFilter();

        intentFilter.addAction(Intent.ACTION_HEADSET_PLUG);



        registerReceiver(headsetChangeListener, intentFilter);



        encodingParameters = getEncodingParameters();



        if (!checkPermissionForCameraAndMicrophone()) {

            requestPermissionForCameraAndMicrophone();

        } else {

            createAudioAndVideoTracks();

            setAccessToken();

        }

    }



    @Override

    public void onRequestPermissionsResult(int requestCode, @NonNull String[] permissions, @NonNull int[] grantResults) {

        if (requestCode == CAMERA_MIC_PERMISSION_REQUEST_CODE) {

            if(checkPermissionForCameraAndMicrophone()) {

                createAudioAndVideoTracks();

                setAccessToken();

            } else {

                Toast.makeText(this,

                        "Camera and Microphone permissions needed. Please allow in App Settings for additional functionality.",

                        Toast.LENGTH_LONG).show();



                AlertDialog.Builder builder = new AlertDialog.Builder(VideoActivity.this);



                builder.setCancelable(false);

                builder.setMessage("App requires permission for camera and microphone. Please allow in App Settings");

                builder.setTitle("Permission");



                builder.setNegativeButton("No", new DialogInterface.OnClickListener() {

                    @Override

                    public void onClick(DialogInterface dialogInterface, int i) {

                        if(alertDialog != null)

                            alertDialog.hide();

                        onBackPressed();

                    }

                });



                builder.setPositiveButton("Go to settings", new DialogInterface.OnClickListener() {

                    @Override

                    public void onClick(DialogInterface dialogInterface, int i) {

                        if(alertDialog != null)

                            alertDialog.hide();

                        Intent intent = new Intent();

                        intent.setAction(Settings.ACTION_APPLICATION_DETAILS_SETTINGS);

                        Uri uri = Uri.fromParts("package", getPackageName(), null);

                        intent.setData(uri);

                        startActivity(intent);

                    }

                });



                alertDialog = builder.create();

                alertDialog.show();



            }

        }

    }



    @Override

    protected void onResume() {

        super.onResume();



        if(isHeadphonesPlugged()) {

            setSpeakerOff();

        } else {

            setSpeakerOn();

        }



        if(checkPermissionForCameraAndMicrophone()) {

            createAudioAndVideoTracks();

        }



        if(room == null && checkPermissionForCameraAndMicrophone()) {

            setAccessToken();

        }

    }



    @Override

    protected void onPause() {

        super.onPause();

    }



    @Override

    protected void onDestroy() {

        unregisterReceiver(headsetChangeListener);



        if (localAudioTrack != null) {

            localAudioTrack.release();

            localAudioTrack = null;

        }

        if (localVideoTrack != null) {

            if (localParticipant != null) {

                localParticipant.unpublishTrack(localVideoTrack);

            }

            localVideoTrack.release();

            localVideoTrack = null;

        }



        if (room != null && room.getState() != RoomState.DISCONNECTED) {

            room.disconnect();

            disconnectedFromOnDestroy = true;

        } else {

            if(VideoExtension.extensionContext != null) {

                VideoExtension.extensionContext = null;

            }

        }



        super.onDestroy();

    }



    public void clickHandler(View view) {

        if(view.getId() == hangupBtn.getId()) {

            onBackPressed();

        }

    }



    private boolean checkPermissionForCameraAndMicrophone(){

        if(Build.VERSION.SDK_INT < Build.VERSION_CODES.M) {

            return true;

        }

        int resultCamera = checkPermission(Manifest.permission.CAMERA, android.os.Process.myPid()

                , android.os.Process.myUid());

        int resultMic = checkPermission(Manifest.permission.RECORD_AUDIO, android.os.Process.myPid()

                , android.os.Process.myUid());

        return resultCamera == PackageManager.PERMISSION_GRANTED &&

                resultMic == PackageManager.PERMISSION_GRANTED;

    }



    private void requestPermissionForCameraAndMicrophone(){

        if(Build.VERSION.SDK_INT < Build.VERSION_CODES.M) {

            return;

        }

        requestPermissions(new String[]{Manifest.permission.CAMERA, Manifest.permission.RECORD_AUDIO}

                , CAMERA_MIC_PERMISSION_REQUEST_CODE);

    }



    private void createAudioAndVideoTracks() {

        if(localAudioTrack == null) {

            localAudioTrack = LocalAudioTrack.create(this, true, LOCAL_AUDIO_TRACK_NAME);

        }



        if(localVideoTrack == null) {

            if(cameraCapturerCompat == null)

                cameraCapturerCompat = new CameraCapturerCompat(this, getAvailableCameraSource());



            localVideoTrack = LocalVideoTrack.create(this,

                    true,

                    cameraCapturerCompat.getVideoCapturer(),

                    LOCAL_VIDEO_TRACK_NAME);

            thumbnailVideoView.setMirror(true);

            localVideoTrack.addRenderer(thumbnailVideoView);

        }



    }



    private CameraCapturer.CameraSource getAvailableCameraSource() {

        return (CameraCapturer.isSourceAvailable(CameraCapturer.CameraSource.FRONT_CAMERA)) ?

                (CameraCapturer.CameraSource.FRONT_CAMERA) :

                (CameraCapturer.CameraSource.BACK_CAMERA);

    }



    private void setAccessToken() {

        retrieveAccessTokenfromServer();

    }



    private void connectToRoom() {

        configureAudio(true);

        ConnectOptions.Builder connectOptionsBuilder = new ConnectOptions.Builder(accessToken)

                .roomName((roomName.isEmpty()?"HomeWAV-test":roomName));



        if (localAudioTrack != null) {

            connectOptionsBuilder

                    .audioTracks(Collections.singletonList(localAudioTrack));

        }



        if (localVideoTrack != null) {

            connectOptionsBuilder.videoTracks(Collections.singletonList(localVideoTrack));

        }



        connectOptionsBuilder.preferAudioCodecs(Collections.singletonList(audioCodec));

        connectOptionsBuilder.preferVideoCodecs(Collections.singletonList(videoCodec));



        connectOptionsBuilder.encodingParameters(encodingParameters);



        room = Video.connect(this, connectOptionsBuilder.build(), roomListener());

    }



    private <T extends Enum<T>> T getCodecPreference(final Class<T> enumClass, final String codec) {

        return Enum.valueOf(enumClass, codec);

    }



    private AudioCodec getAudioCodecPreference(String audioCodecName) {



        switch (audioCodecName) {

            case IsacCodec.NAME:

                return new IsacCodec();

            case OpusCodec.NAME:

                return new OpusCodec();

            case PcmaCodec.NAME:

                return new PcmaCodec();

            case PcmuCodec.NAME:

                return new PcmuCodec();

            case G722Codec.NAME:

                return new G722Codec();

            default:

                return new OpusCodec();

        }

    }



    private VideoCodec getVideoCodecPreference(String videoCodecName) {



        switch (videoCodecName) {

            case Vp8Codec.NAME:

                return new Vp8Codec();

            case H264Codec.NAME:

                return new H264Codec();

            case Vp9Codec.NAME:

                return new Vp9Codec();

            default:

                return new Vp8Codec();

        }

    }



    private EncodingParameters getEncodingParameters() {

        return new EncodingParameters(DEFAULT_MAX_SENDER_AUDIO_BITRATE, DEFAULT_MAX_SENDER_VIDEO_BITRATE);

    }



    private void addRemoteParticipant(RemoteParticipant remoteParticipant) {

        if (!remoteParticipantIdentity.isEmpty()) {

            return;

        }

        remoteParticipantIdentity = remoteParticipant.getIdentity();



        if(VideoExtension.extensionContext != null) {

            VideoExtension.extensionContext.dispatchStatusEventAsync(EVENT_PARTICIPANT_CONNECTED

                    , remoteParticipantIdentity);

        }



        if (remoteParticipant.getRemoteVideoTracks().size() > 0) {

            RemoteVideoTrackPublication remoteVideoTrackPublication =

                    remoteParticipant.getRemoteVideoTracks().get(0);



            if (remoteVideoTrackPublication.isTrackSubscribed()) {

                addRemoteParticipantVideo(remoteVideoTrackPublication.getRemoteVideoTrack());

            }

        }



        remoteParticipant.setListener(remoteParticipantListener());

    }



    private void addRemoteParticipantVideo(VideoTrack videoTrack) {

        primaryVideoView.setMirror(false);

        videoTrack.addRenderer(primaryVideoView);



        if(VideoExtension.extensionContext != null) {

            VideoExtension.extensionContext.dispatchStatusEventAsync(EVENT_PARTICIPANT_VIDEO_TACK_ADDED

                    , remoteParticipantIdentity);

        }

    }



    private void removeRemoteParticipant(RemoteParticipant remoteParticipant) {

        if (!remoteParticipant.getIdentity().equals(remoteParticipantIdentity)) {

            return;

        }



        if(VideoExtension.extensionContext != null) {

            VideoExtension.extensionContext.dispatchStatusEventAsync(EVENT_PARTICIPANT_DISCONNECTED

                    , remoteParticipantIdentity);

        }



        if (!remoteParticipant.getRemoteVideoTracks().isEmpty()) {

            RemoteVideoTrackPublication remoteVideoTrackPublication =

                    remoteParticipant.getRemoteVideoTracks().get(0);



            if (remoteVideoTrackPublication.isTrackSubscribed()) {

                removeParticipantVideo(remoteVideoTrackPublication.getRemoteVideoTrack());

            }

        }



        remoteParticipantIdentity = "";



        onBackPressed();

    }



    private void removeParticipantVideo(VideoTrack videoTrack) {

        videoTrack.removeRenderer(primaryVideoView);



        if(VideoExtension.extensionContext != null) {

            VideoExtension.extensionContext.dispatchStatusEventAsync(EVENT_PARTICIPANT_VIDEO_TRACK_REMOVED

                    , remoteParticipantIdentity);

        }

    }



    private Room.Listener roomListener() {

        return new Room.Listener() {

            @Override

            public void onConnected(Room room) {

                localParticipant = room.getLocalParticipant();

                localParticipant.publishTrack(localVideoTrack);

                localParticipant.setEncodingParameters(encodingParameters);



                if(VideoExtension.extensionContext != null) {

                    VideoExtension.extensionContext.dispatchStatusEventAsync(EVENT_CALL_CONNECTED, roomName);

                }



                for (RemoteParticipant remoteParticipant : room.getRemoteParticipants()) {

                    addRemoteParticipant(remoteParticipant);

                    break;

                }

            }



            @Override

            public void onConnectFailure(Room room, TwilioException e) {

                configureAudio(false);

                if(VideoExtension.extensionContext != null) {

                    VideoExtension.extensionContext.dispatchStatusEventAsync(EVENT_CALL_FAILED, roomName);

                }

                onBackPressed();

            }



            @Override

            public void onDisconnected(Room room, TwilioException e) {

                if(localVideoTrack != null && localParticipant != null)

                    localParticipant.unpublishTrack(localVideoTrack);

                localParticipant = null;

                VideoActivity.this.room = null;



                if (!disconnectedFromOnDestroy) {

                    configureAudio(false);

                }

                setVolumeControlStream(AudioManager.STREAM_SYSTEM);

                if(VideoExtension.extensionContext != null) {

                    VideoExtension.extensionContext.dispatchStatusEventAsync(EVENT_CALL_DISCONNECTED, roomName);

                    VideoExtension.extensionContext = null;

                }



                if (!disconnectedFromOnDestroy) {

                    onBackPressed();

                }

            }



            @Override

            public void onParticipantConnected(Room room, RemoteParticipant remoteParticipant) {

                addRemoteParticipant(remoteParticipant);



            }



            @Override

            public void onParticipantDisconnected(Room room, RemoteParticipant remoteParticipant) {

                removeRemoteParticipant(remoteParticipant);

            }



            @Override

            public void onRecordingStarted(Room room) {



            }



            @Override

            public void onRecordingStopped(Room room) {



            }

        };

    }



    private RemoteParticipant.Listener remoteParticipantListener() {

        return new RemoteParticipant.Listener() {

            @Override

            public void onAudioTrackPublished(RemoteParticipant remoteParticipant,

                                              RemoteAudioTrackPublication remoteAudioTrackPublication) {



            }



            @Override

            public void onAudioTrackUnpublished(RemoteParticipant remoteParticipant,

                                                RemoteAudioTrackPublication remoteAudioTrackPublication) {



            }



            @Override

            public void onDataTrackPublished(RemoteParticipant remoteParticipant,

                                             RemoteDataTrackPublication remoteDataTrackPublication) {



            }



            @Override

            public void onDataTrackUnpublished(RemoteParticipant remoteParticipant,

                                               RemoteDataTrackPublication remoteDataTrackPublication) {



            }



            @Override

            public void onVideoTrackPublished(RemoteParticipant remoteParticipant,

                                              RemoteVideoTrackPublication remoteVideoTrackPublication) {



            }



            @Override

            public void onVideoTrackUnpublished(RemoteParticipant remoteParticipant,

                                                RemoteVideoTrackPublication remoteVideoTrackPublication) {



            }



            @Override

            public void onAudioTrackSubscribed(RemoteParticipant remoteParticipant,

                                               RemoteAudioTrackPublication remoteAudioTrackPublication,

                                               RemoteAudioTrack remoteAudioTrack) {

                if(VideoExtension.extensionContext != null) {

                    VideoExtension.extensionContext.dispatchStatusEventAsync(EVENT_PARTICIPANT_AUDIO_TRACK_ADDED

                            , remoteParticipant.getIdentity());

                }

            }



            @Override

            public void onAudioTrackUnsubscribed(RemoteParticipant remoteParticipant,

                                                 RemoteAudioTrackPublication remoteAudioTrackPublication,

                                                 RemoteAudioTrack remoteAudioTrack) {

                if(VideoExtension.extensionContext != null) {

                    VideoExtension.extensionContext.dispatchStatusEventAsync(EVENT_PARTICIPANT_AUDIO_TRACK_REMOVED

                            , remoteParticipant.getIdentity());

                }

            }



            @Override

            public void onDataTrackSubscribed(RemoteParticipant remoteParticipant,

                                              RemoteDataTrackPublication remoteDataTrackPublication,

                                              RemoteDataTrack remoteDataTrack) {



            }



            @Override

            public void onDataTrackUnsubscribed(RemoteParticipant remoteParticipant,

                                                RemoteDataTrackPublication remoteDataTrackPublication,

                                                RemoteDataTrack remoteDataTrack) {



            }



            @Override

            public void onVideoTrackSubscribed(RemoteParticipant remoteParticipant,

                                               RemoteVideoTrackPublication remoteVideoTrackPublication,

                                               RemoteVideoTrack remoteVideoTrack) {



            }



            @Override

            public void onVideoTrackUnsubscribed(RemoteParticipant remoteParticipant,

                                                 RemoteVideoTrackPublication remoteVideoTrackPublication,

                                                 RemoteVideoTrack remoteVideoTrack) {



            }



            @Override

            public void onAudioTrackEnabled(RemoteParticipant remoteParticipant,

                                            RemoteAudioTrackPublication remoteAudioTrackPublication) {



            }



            @Override

            public void onAudioTrackDisabled(RemoteParticipant remoteParticipant,

                                             RemoteAudioTrackPublication remoteAudioTrackPublication) {



            }



            @Override

            public void onVideoTrackEnabled(RemoteParticipant remoteParticipant,

                                            RemoteVideoTrackPublication remoteVideoTrackPublication) {



            }



            @Override

            public void onVideoTrackDisabled(RemoteParticipant remoteParticipant,

                                             RemoteVideoTrackPublication remoteVideoTrackPublication) {



            }



            @Override

            public void onAudioTrackSubscriptionFailed(RemoteParticipant remoteParticipant

                    , RemoteAudioTrackPublication remoteAudioTrackPublication, TwilioException twilioException) {



            }



            @Override

            public void onVideoTrackSubscriptionFailed(RemoteParticipant remoteParticipant

                    , RemoteVideoTrackPublication remoteVideoTrackPublication, TwilioException twilioException) {



            }



            @Override

            public void onDataTrackSubscriptionFailed(RemoteParticipant remoteParticipant

                    , RemoteDataTrackPublication remoteDataTrackPublication, TwilioException twilioException) {



            }

        };

    }



    private void retrieveAccessTokenfromServer() {

        if(accessToken.isEmpty()) {

            Toast.makeText(VideoActivity.this,

                    "Failed to retrieve access token", Toast.LENGTH_LONG)

                    .show();

        } else {

            connectToRoom();

        }

    }



    private void configureAudio(boolean enable) {

        if (enable) {

            previousAudioMode = audioManager.getMode();



            requestAudioFocus();



            audioManager.setMode(AudioManager.MODE_IN_COMMUNICATION);

           

            previousMicrophoneMute = audioManager.isMicrophoneMute();

            audioManager.setMicrophoneMute(false);

        } else {

            audioManager.setMode(previousAudioMode);

            audioManager.abandonAudioFocus(null);

            audioManager.setMicrophoneMute(previousMicrophoneMute);

        }

    }



    private void requestAudioFocus() {

        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {

            AudioAttributes playbackAttributes = new AudioAttributes.Builder()

                    .setUsage(AudioAttributes.USAGE_VOICE_COMMUNICATION)

                    .setContentType(AudioAttributes.CONTENT_TYPE_SPEECH)

                    .build();

            AudioFocusRequest focusRequest =

                    new AudioFocusRequest.Builder(AudioManager.AUDIOFOCUS_GAIN_TRANSIENT)

                            .setAudioAttributes(playbackAttributes)

                            .setAcceptsDelayedFocusGain(true)

                            .setOnAudioFocusChangeListener(

                                    new AudioManager.OnAudioFocusChangeListener() {

                                        @Override

                                        public void onAudioFocusChange(int i) { }

                                    })

                            .build();

            audioManager.requestAudioFocus(focusRequest);

        } else {

            audioManager.requestAudioFocus(null, AudioManager.STREAM_VOICE_CALL,

                    AudioManager.AUDIOFOCUS_GAIN_TRANSIENT);

        }

    }



    private boolean isHeadphonesPlugged() {

        Boolean pluggedIn = false;

        if(Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {

            AudioDeviceInfo[] audioDevices = audioManager.getDevices(AudioManager.GET_DEVICES_ALL);

            for(AudioDeviceInfo deviceInfo : audioDevices){

                if(deviceInfo.getType()==AudioDeviceInfo.TYPE_WIRED_HEADPHONES

                        || deviceInfo.getType()== AudioDeviceInfo.TYPE_WIRED_HEADSET){

                    pluggedIn = true;

                    break;

                }

            }

        } else {

            pluggedIn = audioManager.isWiredHeadsetOn();

        }





        return pluggedIn;

    }



    private void setSpeakerOn() {

        audioManager.setSpeakerphoneOn(true);

    }



    private void setSpeakerOff() {

        audioManager.setSpeakerphoneOn(false);

    }



    private BroadcastReceiver headsetChangeListener = new BroadcastReceiver() {

        @Override

        public void onReceive(Context context, Intent intent) {

            if (intent.getAction().equals(Intent.ACTION_HEADSET_PLUG)) {

                int state = intent.getIntExtra("state", -1);

                switch (state) {

                    case 0:

                        setSpeakerOn();

                        break;

                    case 1:

                        setSpeakerOff();

                        break;

                }

            }

        }

    };

}

activity_video.xml


<?xml version="1.0" encoding="utf-8"?>

<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"

    xmlns:app="http://schemas.android.com/apk/res-auto"

    xmlns:tools="http://schemas.android.com/tools"

    android:layout_width="match_parent"

    android:layout_height="match_parent"

    android:keepScreenOn="true"

    tools:context="ijasnahamed.twilio.video.views.VideoActivity">



    <com.twilio.video.VideoView

            android:id="@+id/LocalView"

            android:layout_width="96dp"

            android:layout_height="96dp"

            android:layout_margin="16dp"

            android:layout_alignParentRight="true"

            android:layout_alignParentBottom="true"/>



    <com.twilio.video.VideoView

        android:id="@+id/RemoteView"

        android:layout_width="match_parent"

        android:layout_height="match_parent"

        android:layout_gravity="center"/>



    <Button

        android:layout_width="wrap_content"

        android:layout_height="wrap_content"

        android:id="@+id/DisconnectBtn"

        android:text="Disconnect"

        android:layout_alignParentLeft="true"

        android:layout_alignParentBottom="true"

        android:layout_margin="10dp"

        android:background="#f45342"

        android:textColor="#fff"

        android:paddingLeft="5dp"

        android:paddingRight="5dp"

        android:onClick="clickHandler"/>



</RelativeLayout>

You have noticed in call method in VideoFunction class(check Step 6), we are accessing different view id using freContext and passing with out activity intent without being calling from activity. This is because, when we export our android library, adobe cannot access app's R file. So extension will crash when we call R.id method from activity.

Also, we are getting some layout file and passing with activity intent. I will explain why we did this in next part.

Build and generate your android library

Android Studio doesn't allow to export code to jar directly. So close our project in Studio IDE and open in IntelliJ IDE.

IntelliJ may give you an error that, we have to use Android Studio 3. This is because, Twilio Video SDK was build with Android Studio 3. To suppress this error, open gradle.properties file and add below code to it.

android.injected.build.model.only.versioned=3

Now, build the project.

Next, we need to export our code in jar format. Goto File->Project Structure in main toolbar. This will open a dialog. Under Project settings, select Artifacts.

Create a new artifact by clicking + icon. When you click add button, select Jar. Under Jar, select From modules with dependency.

Now our artifact is created and it has our code with all our  dependencies. From output layout tab of our artifact settings, remove FlashRuntimeExtensions.jar, because our adobe AIR app already has this. And Apply this change.

Next, Go to Build menu in main toolbar and select Build artifacts. Now a small popup is displayed with all artifacts. Select your artifact and select Build. Now, our android library build process will start and will take some time to get our final jar file.

Please, create a new artifact if any of your dependency changes because artifact takes all dependency at its creation time, not on each artifact build.

Our build jar file will be available in Builds/classes/artifacts/{artifact_name} directory from android project root directory.


You can download a sample android library from ijasnahamed > Twilio Video ANE > Android Library.


Now lets integrate both wrapper and Android library to build our final extension from https://www.ijasnahamed.in/2018/07/twilio-video-adobe-native-extension-part-3.html.

Popular posts from this blog

Setup Asterisk 13 with FreePBX 13 in CentOS 7

Launch CentOS 7 AWS Ec2 InstanceLog in to your aws consoleGo to ec2 management page and click Launch Instance on Instance pageIn Choose AMI step, go to AWS MarketPlace tab and search CentOS 7 on search field. List of centos 7 ami's will be available.Select CentOS 7 (x86_64) ami which is free tier eligible. Select your instance type and click configure instance. Keep default values from Configure Instance to Add Tags stepsIn configure security group, create a security group which allow minimum ports openSSH : port 22HTTP : port 80Custom UDP Rule : 10000-20000 (if you are giving RTP ports 10000-20000)Custom UDP Rule : 5060 (ChanSIP port)Custom TCP Rule : 5060 (ChanSIP Port)Click Launch Instance. Select your key file and accept terms to launch instance.
Setup CentOS Server SSH into newly created CentOS server with username centos and your key file.Update all existing packagessudo yum update -yNow start http service(you can access your server via browser by going to your server ip addres…

Implementing Client Side WebRTC using Sipml5 javascript

Step 1:

Download and require Sipml5 library function.

Step 2:

Initialize sipml5 Engine in your web page :

var readyCallback = function(e) {
// function called when sipml is successfully initialised.
createSipStack(); // calling this function to create sip stack(see below)
};

var errorCallback = function(e) {
// function called when error occured during sipml initialisation.
};

SIPml.init(readyCallback, errorCallback);

Step 3:

Create Sip Stack :-
Sip Stack is an object that must be created before making/receiving call and sms. Creating Sip stack is an asynchronous process, so you need to create an event listener function to get state change notification.

var sipStack;

function EventListener(e) {

/*
* e.type ;type of event listener
* e.session ; current event session
* e.getSipResponseCode() ; event response code
* e.description ; event description
*/

if(e.type == 'started'){
// successfully started the stack.
register();
} else if(e.type == 'i_new_call'){
// when new incoming call comes.
      …

Multiple file upload using ajax with progress bar

Uploading multiple files using ajax makes a pleasant feeling to the user. It makes even more happier if upload progress status is printed with percentage.

            In this blog, you will make a file upload form using ajax with file upload progress bar.

Prerequisites :

1) Javascript supported browser
2) LAMP Server / XAMPP Server

Code :

index.html

<!DOCTYPE html>

<html>

<head>

    <title>Multiple File Upload using Ajax</title>

    <link rel="stylesheet" type="text/css" href="style.css">

    <script src="//code.jquery.com/jquery-1.10.2.min.js"></script>

</head>

<body>

    <div>

        <form action="action.php" method="post" enctype="multipart/form-data" id="multiple-upload-form">

            <input type="button" id="select-file-btn" value="Select Files" onclick="document.getElementById(…