Install SSL Letsencrypt on Wowza Server HLS Stream

I am running webserver / domain on HTTPS and Wowza Media Streaming Server on HTTP (Non-SSL) so I am getting Error for LIVE / VOD HLS stream Files “Cannot load M3U8: Unable to fetch HTTP resource over HTTPS”

HTTP stream
http://origin.domain.com:1935/live/myStream/playlist.m3u8

HTTPS stream – Needs to setup
https://origin.domain.com:1443/live/myStream/playlist.m3u8

Install Let’s Encrypt SSL Certificates
Free SSL Certbot

=> As I have installed SSL on NGINX … I am running NGINX webserver on 443 (HTTPS) … but wowza is not configured for SSL still

Try to find SSL key in Let’s Encrypt Certificate Directory

# ls /etc/letsencrypt/live
# ls /etc/letsencrypt/live/origin.domain.com
cert.pem chain.pem fullchain.pem privkey.pem README

cert.pem: Server Certificate
chain.pem: Root and Intermediate Certificates
fullchain.pem: Combination of Server, Root and Intermediate Certificates (replaces cert.pem and chain.pem)
privkey.pem: Private Key (don’t share)

# stat /etc/letsencrypt/live/origin.domain.com/fullchain.pem
File: ‘/etc/letsencrypt/live/origin.domain.com/fullchain.pem’ -> ‘../../archive/origin.domain.com/fullchain3.pem’
Size: 50 Blocks: 0 IO Block: 4096 symbolic link
Device: ca01h/51713d Inode: 2754252 Links: 1

Wowza Configuration

Special Thanks to Robymus who made java converter file “wowza-letsencrypt-converter” to convert SSL to an JKS file.

# cd /usr/local/WowzaStreamingEngine/lib
# wget https://github.com/robymus/wowza-letsencrypt-converter/releases/download/v0.1/wowza-letsencrypt-converter-0.1.jar

– The letsencrypt-live-path parameter defaults to /etc/letsencrypt/live

Make sure you have Java 8 installed

# apt-get install oracle-java8-installer

# cd /usr/local/WowzaStreamingEngine/lib
# java -jar wowza-letsencrypt-converter-0.1.jar -v /usr/local/WowzaStreamingEngine/conf/ /etc/letsencrypt/live/

Make sure you will have below files in /usr/local/WowzaStreamingEngine/conf/

– origin.domain.com.jks
– jksmap.txt

– File jksmap.txt have domain to keystore mapping will be used in the VHost.xml of Wowza Streaming Engine.
– JKS password will be ‘secret’.

Now update Wowza file /usr/local/WowzaStreamingEngine/conf/VHost.xml for Wowza HLS M3U8 file

Open Wowza VHost.xml and search 443 HostPort and comment out the <!– before HostPort and –> at the end of HostPort

I am using port 443 already for NGINX SSL HTTPS … so now I am using port 1443 for Wowza HLS M3U8

<!-- 443 with SSL -->
            <HostPort>
                <Name>Default SSL Streaming</Name>
                <Type>Streaming</Type>
                <ProcessorCount>${com.wowza.wms.TuningAuto}</ProcessorCount>
                <IpAddress>*</IpAddress>
                <Port>1443</Port>
                <HTTPIdent2Response></HTTPIdent2Response>
                <SSLConfig>
                    <KeyStorePath>${com.wowza.wms.context.VHostConfigHome}/conf/origin.domain.com.jks</KeyStorePath>
                    <KeyStorePassword>secret</KeyStorePassword>
                    <KeyStoreType>JKS</KeyStoreType>
                    <DomainToKeyStoreMapPath></DomainToKeyStoreMapPath>
                    <SSLProtocol>TLS</SSLProtocol>
                    <Algorithm>SunX509</Algorithm>
                    <CipherSuites></CipherSuites>
                    <Protocols></Protocols>
                </SSLConfig>
                <SocketConfiguration>
                    <ReuseAddress>true</ReuseAddress>
                    <ReceiveBufferSize>65000</ReceiveBufferSize>
                    <ReadBufferSize>65000</ReadBufferSize>
                    <SendBufferSize>65000</SendBufferSize>
                    <KeepAlive>true</KeepAlive>
                    <AcceptorBackLog>100</AcceptorBackLog>
                </SocketConfiguration>
                <HTTPStreamerAdapterIDs>cupertinostreaming,smoothstreaming,sanjosestreaming,dvrchunkstreaming,mpegdashstreaming</HTTPStreamerAdapterIDs>
                <HTTPProviders>
                    <HTTPProvider>
                        <BaseClass>com.wowza.wms.http.HTTPCrossdomain</BaseClass>
                        <RequestFilters>*crossdomain.xml</RequestFilters>
                        <AuthenticationMethod>none</AuthenticationMethod>
                    </HTTPProvider>
                    <HTTPProvider>
                        <BaseClass>com.wowza.wms.http.HTTPClientAccessPolicy</BaseClass>
                        <RequestFilters>*clientaccesspolicy.xml</RequestFilters>
                        <AuthenticationMethod>none</AuthenticationMethod>
                    </HTTPProvider>
                    <HTTPProvider>
                        <BaseClass>com.wowza.wms.http.HTTPProviderMediaList</BaseClass>
                        <RequestFilters>*jwplayer.rss|*jwplayer.smil|*medialist.smil|*manifest-rtmp.f4m</RequestFilters>
                        <AuthenticationMethod>none</AuthenticationMethod>
                    </HTTPProvider>
                    <HTTPProvider>
                        <BaseClass>com.wowza.wms.http.HTTPServerVersion</BaseClass>
                        <RequestFilters>*</RequestFilters>
                        <AuthenticationMethod>none</AuthenticationMethod>
                    </HTTPProvider>
                </HTTPProviders>
            </HostPort>

Now Restart Wowza Server

# service WowzaStreamingEngine restart

Make sure you have also opened port 1443 on your firewall.

Now Wowza HLS M3U8 will work like HTTPS stream
https://origin.domain.com:1443/live/myStream/playlist.m3u8

Convert AWS DynamoDB Table JSON to Simple PHP Array or JSON

Amazon DynamoDB support these datatype for data

– String
– Binary
– Number
– StringSet
– NumberSet
– BinarySet
– Map
– List
– Boolean
– Null

AWS provides command line tool to SCAN DynamoDB Tables and export as JSON file

# /usr/local/bin/aws dynamodb scan –table-name Users –query ‘Items[*]’ –region us-east-1 –output json

Amazon DynamoDB Formatted JSON using JQ

But AWS DynamoDB exports Table in special JSON format … we can’t import in MySQL without process it to normal JSON file 🙁

AWS DynamoDB JSON

<?php

echo "<pre>";
$tmp = "Users.json";

function aws_dd_json_converter($old) {

    $new = array(); $new1 = array();

	foreach($old as $k=>$v) {
	   foreach($v as $k1=>$v1) {	
       $new[$k1] = isset($v[$k1]['S'])?$v[$k1]['S']:$v[$k1]['N'];  // we can add more datatype here
       }	
	   $new1[] = $new;	
	}
	return $new1;
}

echo "<strong>Original AWS DynamoDB JSON</strong><br>";
echo $line = file_get_contents($tmp);
$array = json_decode($line,true);

$array = aws_dd_json_converter($array);
echo "<br><strong>Simple PHP Array for AWS DynamoDB JSON</strong><br>";
print_r($array);

echo "<br><strong>Simple JSON for AWS DynamoDB JSON</strong><br>";
print(json_encode($array, JSON_PRETTY_PRINT));

?>

Install SSH2 Extension for PHP 7 on CentOS 7

Install gcc, php71w-devel, libssh2 and libssh2-devel on CentOS 7
# yum install gcc php71w-devel libssh2 libssh2-devel

Download php7 pecl-networking-ssh2
# wget https://github.com/Sean-Der/pecl-networking-ssh2/archive/php7.zip

Unzip and Change folder
# unzip php7.zip
# cd pecl-networking-ssh2-php7

Building environment for SSH2 extension
# phpize

# ./configure

Make SSH2 extension make sure build success
# make

Install SSH2 extension
# make install

Write SSH2 extension in PHP 7 extension directory
# vi /etc/php.d/ssh2.ini
extension=ssh2.so

Restart NGINX server and php-fpm
# systemctl restart nginx
# systemctl restart php-fpm

SSH2 PHP7 CentOS7

SSH2 module enabled in PHP 7
# php -m
[PHP Modules]
apc
apcu
bz2
calendar
Core
ctype
curl
date
dom
exif
fileinfo
filter
ftp
gd
gettext
hash
iconv
intl
json
libxml
mbstring
mcrypt
mysqli
mysqlnd
openssl
pcntl
pcre
PDO
pdo_mysql
pdo_sqlite
Phar
posix
readline
Reflection
session
shmop
SimpleXML
sockets
SPL
sqlite3
ssh2
standard
sysvmsg
sysvsem
sysvshm
tokenizer
wddx
xml
xmlreader
xmlwriter
xsl
zip
zlib

[Zend Modules]

Attach And Mount EBS Volume To EC2 Linux Instance

AWS allows to create / attach / mount new EBS volumes to instances for extra storage. Use lsblk command to view available disk devices and other mount points.

# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 8.0G 1.6G 6.5G 20% /

– Go to EC2 Console –> Volumes to create new volume of required size and type.

– Select the last created volume and select the “attach volume” to attach to the ec2 instance.

– Now, SSH login to ec2 instance and find list of available disks or mounts.

# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 8G 0 disk
└─xvda1 202:1 0 8G 0 part /
xvdf 202:80 0 16G 0 disk

Check the volume has data or empty
# file -s /dev/xvdf

Format the volume to ext4 file system type
# mkfs -t ext4 /dev/xvdf

Mount the volume to “/mnt/dir/” directory
# mkdir /mnt/dir/
# mount /dev/xvdf /mnt/dir/ -t ext4

# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 8.0G 1.6G 6.5G 20% /
/dev/xvdf 16G 45M 15G 1% /mnt/dir

# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 8G 0 disk
└─xvda1 202:1 0 8G 0 part /
xvdf 202:80 0 16G 0 disk /mnt/dir

EBS Automount On Reboot using fstab
# cp /etc/fstab /etc/fstab.bak

Write FSTAB file
# vi /etc/fstab
/dev/xvdf /mnt/dir ext4 defaults,nofail

Check errors
# mount -a

How to unmount volume??
# umount /dev/xvdf

Help: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-using-volumes.html

Install Ruby Slate NodeJS on CentOS 7

Slate is Beautiful static documentation for your API

Slate

Required

– Linux or OS X or Windows
– Ruby version 2.3.1 or newer
– NodeJS
– Bundler — gem install bundler

# cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)

Install Ruby 2.4 – CentOS 7

# yum update
# yum groupinstall “Development Tools”
# yum install epel-release

# yum install readline-devel zlib-devel libffi-devel libyaml-devel openssl-devel sqlite-devel

RVM (Ruby Version Manager) Allow you to install and manage multiple ruby environments.

# curl -sSL https://rvm.io/mpapis.asc | gpg --import -
# curl -L get.rvm.io | bash -s stable

# source /etc/profile.d/rvm.sh
# rvm reload

# rvm requirements run

Show available ruby versions
# rvm list known

Install Ruby 2.4

# rvm install 2.4
# rvm use 2.4 --default

Install Ruby Bundler

# gem install bundler

Install NodeJS for JavaScript Run-time Environment

# yum install nodejs npm

# cd /home/centos

Fork / Clone repository

# git clone https://github.com/svnlabs/slate.git

# cd slate

Allow / Open firewall port 4567 for Slate

# systemctl status firewalld

Like firewall-cmd –add-port=[YOUR PORT]/tcp

# firewall-cmd --permanent --add-port=4567/tcp

# systemctl restart firewalld

Run Slate locally

# bundle install
# bundle exec middleman server

Open URL in Browser http://server-ip-address:4567/#introduction

Now edit source files here “/home/centos/slate” as you need 😉

PHP Parse HLS M3U8 TS Links

Input

encoded-03-31-16-thu-jun-2017.m3u8

#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=654000,RESOLUTION=568×320,CODECS=”avc1.4d0029,mp4a.40.2″
1080p/encoded-03-31-16-thu-jun-2017.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2849000,RESOLUTION=568×320,CODECS=”avc1.42001f,mp4a.40.2″
720p/encoded-03-31-16-thu-jun-2017.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1033000,RESOLUTION=568×320,CODECS=”avc1.42001e,mp4a.40.2″
360p/encoded-03-31-16-thu-jun-2017.m3u8

encoded-03-31-16-thu-jun-2017.360.m3u8

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:13
#EXT-X-KEY:METHOD=AES-128,URI=”encoded-03-31-16-thu-jun-2017.key”,IV=0x5fd83c919c46159f14ec8da38a4d9ca7
#EXTINF:12.046444,
encoded-03-31-16-thu-jun-201700000.ts
#EXTINF:9.000000,
encoded-03-31-16-thu-jun-201700001.ts
#EXTINF:9.000000,
encoded-03-31-16-thu-jun-201700002.ts
#EXTINF:12.000000,
encoded-03-31-16-thu-jun-201700003.ts
.
.
.
#EXT-X-ENDLIST

Code

<?php

function parseHLS($file, $tag=".ts") {
$return = array();
$i = 0;
$handle = fopen($file, "r");
if($handle) {
while(($line = fgets($handle)) !== FALSE) {
if(strpos($line,"EXT-X-STREAM-INF") !== FALSE) {
if ($c=preg_match_all ("/.*?(BANDWIDTH)(.*?)(,)(RESOLUTION)(.*?)(,)/is", $line, $matches)) {
$return['data'][$i]['bandwidth'] = str_replace("=","",$matches[2][0]);
$return['data'][$i]['resolution'] = str_replace("=","",$matches[5][0]);
}
}

if($tag==".ts") {
if(strpos($line,".ts") !== FALSE) {
$return['data'][$i]['url'] = str_replace(array("\r","\n"),"",$line);
$i++;
}
}

if($tag==".key") {
if(strpos($line,".key") !== FALSE) {  

preg_match( '/URI="([^"]*)"/i', $line, $array );
//print_r( $array[1] ) ;

//$return['data'][$i]['url'] = str_replace(array("\r","\n"),"",$line);
$return['data'][$i]['url'] = str_replace(array("\r","\n"),"",$array[1]);

$i++;
}
}

if($tag==".m3u8") {
if(strpos($line,".m3u8") !== FALSE) {
$return['data'][$i]['url'] = str_replace(array("\r","\n"),"",$line);
$i++;
}
}

}
fclose($handle);
}
return $return;
}


echo "<pre>";

// Get other m3u8 files from m3u8 master manifest 
$u = parseHLS("encoded-03-31-16-thu-jun-2017.m3u8", ".m3u8");

print_r($u);

// Get TS segment files from m3u8 manifest 
$u = parseHLS("encoded-03-31-16-thu-jun-2017.360.m3u8", ".ts");

print_r($u);

// Get key file from m3u8 manifest 
$u = parseHLS("encoded-03-31-16-thu-jun-2017.360.m3u8", ".key");

print_r($u);

echo "</pre>";

?>

Output

Array
(
[data] => Array
(
[0] => Array
(
[bandwidth] => 654000
[resolution] => 568x320
[url] => 1080p/encoded-03-31-16-thu-jun-2017.m3u8
)

[1] => Array
(
[bandwidth] => 2849000
[resolution] => 568x320
[url] => 720p/encoded-03-31-16-thu-jun-2017.m3u8
)

[2] => Array
(
[bandwidth] => 1033000
[resolution] => 568x320
[url] => 360p/encoded-03-31-16-thu-jun-2017.m3u8
)

)

)

Array
(
[data] => Array
(
[0] => Array
(
[url] => encoded-03-31-16-thu-jun-201700000.ts
)

[1] => Array
(
[url] => encoded-03-31-16-thu-jun-201700001.ts
)

[2] => Array
(
[url] => encoded-03-31-16-thu-jun-201700002.ts
)

[3] => Array
(
[url] => encoded-03-31-16-thu-jun-201700003.ts
)
....
.....
)
)

Array
(
[data] => Array
(
[0] => Array
(
[url] => encoded-03-31-16-thu-jun-2017.key
)

)

)

PHP in Amazon Lambda

Clone or Download this GitHub Repo
https://github.com/svnlabs/aws-lambda-php-template

Make new folder lambda in current working directory of EC2 server
# mkdir lambda

Upload 2 files from zip to Amazon EC2 server Instance in folder lambda

lambda
– index.js
– php (binary)

index.js

'use strict';
exports.handler = function(event, context, callback) {
var exec = require('child_process').exec;

  exec('./php -v', function (error, stdout, stderr) {
    if(stdout){
      console.log('stdout: ' + stdout);
    }
    if(stderr){
      console.log('stderr: ' + stderr);
    }
    if (error !== null) {
      console.log('Exec error: ' + error);
    }
    context.succeed(stdout);
    console.log(stdout);
  });
};

Go inside folder lambda

# cd lambda
# zip -r ../lambda.zip *

Copy lambda.zip for AWS S3 bucket

# aws s3 cp ../lambda.zip s3://bucket-name/

How to install AWS CLI?

Create new Lambda Function from AWS Console with NodeJS Runtime*

Name*: phpv
Runtime*: Node.js 4.3
Code entry type: Upload a file from Amazon S3
S3 link URL*: https://s3.amazonaws.com/bucket-name/lambda.zip

Handler*: index.handler
Role*: Lambda Basic Execution

All other info will be default**

Save & Test

“PHP 5.6.5 (cli) (built: Jul 21 2015 18:56:08) \nCopyright (c) 1997-2014 The PHP Group\nZend Engine v2.6.0, Copyright (c) 1998-2014 Zend Technologies\n”

Free SSL Certbot

Certbot, It was Let’s Encrypt Client previously. Automatically enable HTTPS on your website with EFF’s Certbot, deploying Let’s Encrypt certificates.

Server Support

– Apache
– Nginx
– Haproxy
– Plesk

OS Support

– Web Hosting Service
– Debian 7 (wheezy)
– Debian 8 (jessie)
– Debian 9 (stretch)
– Debian testing/unstable
– Debian (other)
– Ubuntu 17.04 (zesty)
– Ubuntu 16.10 (yakkety)
– Ubuntu 16.04 (xenial)
– Ubuntu 14.04 (trusty)
– Ubuntu (other)
– Gentoo
– Arch Linux
– Fedora 24+
– CentOS 6
– RHEL 6
– CentOS/RHEL 7
– FreeBSD
– OpenBSD 5.9
– OpenBSD 6.0+
– OpenBSD (other)
– macOS
– Devuan Jessie 1.0
– Devuan (other)
– Other UNIX
– Non-UNIX

Install on Ubuntu 17.04 (zesty) Apache

$ sudo apt-get update
$ sudo apt-get install software-properties-common
$ sudo add-apt-repository ppa:certbot/certbot
$ sudo apt-get update
$ sudo apt-get install python-certbot-apache

Get Started https://certbot.eff.org/#ubuntutzesty-apache

$ sudo certbot --apache

$ sudo certbot --apache certonly

Automating renewal

$ sudo certbot renew --dry-run

$ sudo crontab -e

30 1 * * * /usr/bin/certbot renew --quiet

It will run renewal command at 1:30 am, every day.

Solution: Client with the currently selected authenticator does not support any combination of challenges that will satisfy the CA

$ sudo certbot --authenticator standalone --installer apache -d stream.domain.com --pre-hook "service apache2 stop" --post-hook "service apache2 start"

.
.
.

Congratulations! You have successfully enabled https://stream.domain.com

You should test your configuration at:
https://www.ssllabs.com/ssltest/analyze.html?d=stream.domain.com

PHP Warning: PHP Startup: Unable to load dynamic library /usr/lib64/php/ modules/zip.so

CentOS 7 with PHP

# php --ini
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/zip.so’ – /usr/lib64/php/modules/zip.so: undefined symbol: php_pcre_exec in Unknown on line 0
Configuration File (php.ini) Path: /etc
Loaded Configuration File: /etc/php.ini
Scan for additional .ini files in: /etc/php.d
Additional .ini files parsed: /etc/php.d/10-opcache.ini,
/etc/php.d/20-bcmath.ini,
/etc/php.d/20-bz2.ini,
/etc/php.d/20-calendar.ini,
.
.
.

Install Apache, MySQL, PHP 5.6 on Centos 7

Solution

# yum install php-zip

# service httpd restart
Redirecting to /bin/systemctl restart httpd.service

PHP Check Loaded Modules

# php -m
[PHP Modules]
apc
apcu
bcmath
bz2
calendar
Core
ctype
curl
date
dom
ereg
exif
fileinfo
filter
ftp
gd
gettext
gmp
hash
iconv
igbinary
json
libxml
mbstring
mcrypt
memcache
memcached
mhash
mongo
msgpack
mysql
mysqli
mysqlnd
openssl
pcntl
pcre
PDO
pdo_mysql
pdo_pgsql
pdo_sqlite
pgsql
Phar
posix
readline
recode
redis
Reflection
session
shmop
SimpleXML
sockets
SPL
SQLite
sqlite3
standard
sysvmsg
sysvsem
sysvshm
tidy
tokenizer
wddx
xml
xmlreader
xmlwriter
xsl
Zend OPcache
zip
zlib

[Zend Modules]
Zend OPcache

PHP Check Loaded ini files
# php --ini

# php -v
PHP 5.6.31 (cli) (built: Jul 6 2017 08:06:11)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies

Install Apache, MySQL, PHP 5.6 on Centos 7

Install Apache

# yum update
# yum install httpd
# systemctl start httpd.service

Now Browser your Apache Server http://Server_IP_address/

Enable Apache to start at server boot

# systemctl enable httpd.service

Install Mariadb MySQL

# yum install mariadb-server mariadb
# systemctl start mariadb
# mysql_secure_installation

Enable MySQL to start at server boot

# systemctl enable mariadb.service

Install PHP 5.6

Install EPEL repository

# rpm -Uvh http://vault.centos.org/7.0.1406/extras/x86_64/Packages/epel-release-7-5.noarch.rpm

Note: Make sure you installed “epel-release-7-5.noarch.rpm” else google it for different location …. search on Google “index of” + epel-release-7-5.noarch.rpm

Install remi repository

# rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-7.rpm

Note: Make sure you installed “remi-release-7.rpm” else google it for different location

Enable remi

# yum --enablerepo=remi,remi-php56 install php php-common

Install php 5.6 on Centos 7

# yum --enablerepo=remi,remi-php56 install php-cli php-pear php-pdo php-mysql php-mysqlnd php-pgsql php-sqlite php-gd php-mbstring php-mcrypt php-xml php-simplexml php-curl php-zip

# systemctl restart httpd.service
# chown -R apache:apache /var/www/html/

# chmod -R 775 /var/www/

<?php phpinfo(); ?>

http://Server_IP_address/phpinfo.php