Algorithm¶

Machine learning Model¶

Soft Actor Critic using Stable Baseline(Reinforcement Learning)¶

• I refered this Link for setup.
• This algorithm is mainly based on Reward function.More points are given when robot follows a path and less point is given when it goes out of the track.
• Install Tensorflow 1.10.1 version for this algo and use it in different environment.
• Initially a Variational Auto encoder(VAE) Pretrained agent is used for improving the performance of model along with reward function. Hyperparameters will also affect the performance of model. By changing number of epoch,batch size etc model can be improved.

Linear Model¶

• It is a Supervised algorithm.
• Refer BlockDiagram section for more info.
• Also Refer Keras linear model more info.
• It will mimic the human driving behaviour.

Switching Between Autonomous and Manual Mode¶

• We are using the Donkeycar pacakge in our project.Import the same as shown below.

import donkey as dk
#initialize the vehicle
V = dk.Vehicle()

Check any object detected based on Label Value¶

    #So we initialise the object of Objectdetectionflag class for setting flag variable when object detected 
    V.add(ObjectDetectionflag(),inputs=['cam/image_array'],outputs=['flag'])
    #keeping the negate value of flag variable
    class Negate():
       def __init__(self):
          pass
       def run(self,flag):
          if flag==True:
             return False
          else:
             return True
    # adding the part to the vehicle
    V.add(Negate(),inputs=['flag'],outputs=['Negate'])
    #Use the run_condition parameter to add part to the vehicle or not this is where webcontroller sitches modes
    V.add(ctr, 
          inputs=['cam/image_array'],
          outputs=['user/angle', 'user/throttle', 'user/mode', 'recording'],
          threaded=True,run_condition='flag')

"""
This class is used for Manual mode operations.Right now robot take decision within 1.5 seconds i.e it moves left then again come back to track after travelling through some distance.
"""
class ValueFetching:
       def __init__(self):
         self.start=None
         self.steering=0.45
         self.throttle=0.6
       def run(self):  
          #this should be called for the first time only 
          if m.count==1:
            self.start=datetime.now()
          m.count=255
          endtime=datetime.now()
          print((endtime-self.start).seconds)
          if (endtime-self.start).seconds>1.5:
              m.count=0
              self.steering=0.45
              m.x=False

          if (endtime-self.start).seconds<1.5:
              self.throttle=0.45
              # changing the steering dynamically based on previous value
              self.steering=self.steering-0.04
              m.x=True
          return self.steering,self.throttle,'user',False
    # add the part to the vehicle when run_condition is reverse that is opposite to webcontroller part.
    #Negate class is used for the above mentioned purpose
    V.add(ValueFetching(),outputs=['user/angle', 'user/throttle', 'user/mode', 'recording'],run_condition='Negate')

Choose between Image zmq and ROS Image Publishing¶

• I used both of the methods to send images to rpi. In my opinion Imagezmq method performed well.
• You can choose any of the methods.

import imagezmq
import socket
import time
#create a part or class which publishes image.Replace tcp address of master by address you get using 
#hostname -I on Master
class ImagePublish:
  def __init__(self):
    self.sender = imagezmq.ImageSender(connect_to="tcp://192.168.1.7:5555")
    self.rpiName = socket.gethostname()
    self.imageHub = imagezmq.ImageHub()


  def run(self,image):
    # send the image only if not none and only if sender is ready
    if(image is not None and self.sender is not None):
        print("Connection successful")
        print(self.sender.send_image(self.rpiName,image))



  def shutdown(self):
    pass

import os
import rospy
from std_msgs.msg import String, Int32, Float32
from sensor_msgs.msg import Image
from cv_bridge import CvBridge, CvBridgeError

# this part should be called in manage.py
class RosPublisher(object):
    '''
    A ROS node to pubish to a Image data stream
    '''
    def __init__(self, node_name, channel_name, stream_type=Image, anonymous=True):
        #replace ip with your host address
        os.environ['ROS_IP'] = '192.168.1.4'

        #run roscore in master before running slave
        os.environ['ROS_MASTER_URI'] = 'http://192.168.1.4:11311/'

        self.pub = rospy.Publisher(channel_name, stream_type,queue_size=10)
        self.bridge = CvBridge()
        rospy.init_node(node_name, anonymous=anonymous)

    def run(self, data):
        '''
        only publish when data stream changes.
        '''
        if data is not None and not rospy.is_shutdown():
            # cvbridge is used to convert open cv or numpy image to sensor image suitable with ros msg passing
            self.pub.publish(self.bridge.cv2_to_imgmsg(data, "bgr8"))

Behaviour Clonning¶

• This is a simple CNN followed by deep neural network required for training the images to predict throttle,steering values.

    drop = 0.1

    img_in = Input(shape=input_shape, name='img_in')

    #get the image passed from part initialisation
    x = img_in
   `x = Convolution2D(24, (5,5), strides=(2,2), activation='relu', name="conv2d_1")(x)
    x = Dropout(drop)(x)
    x = Convolution2D(32, (5,5), strides=(2,2), activation='relu', name="conv2d_2")(x)
    x = Dropout(drop)(x)
    x = Convolution2D(64, (5,5), strides=(2,2), activation='relu', name="conv2d_3")(x)
    x = Dropout(drop)(x)
    x = Convolution2D(64, (3,3), strides=(1,1), activation='relu', name="conv2d_4")(x)
    x = Dropout(drop)(x)
    x = Convolution2D(64, (3,3), strides=(1,1), activation='relu', name="conv2d_5")(x)
    x = Dropout(drop)(x)
    #flatten it so that it fits to Deep neural network layer input
    x = Flatten(name='flattened')(x)
    x = Dense(100, activation='relu')(x)
    x = Dropout(drop)(x)
    x = Dense(50, activation='relu')(x)
    #drop some layer helps in performance improvement
    x = Dropout(drop)(x)`

   outputs = []
    #output consists of steering and throttle
    #final model object which is saved as model.h5 file
    for i in range(num_outputs):
        outputs.append(Dense(1, activation='linear', name='n_outputs' + str(i))(x))

    model = Model(inputs=[img_in], outputs=outputs)

• Predicting the steering and throttle based on linear model output which takes realtime image as input.

    img_arr = img_arr.reshape((1,) + img_arr.shape)
    outputs = self.model.predict(img_arr)
    steering = outputs[0]
    throttle = outputs[1]