Tutorials

How to Use Perinno-Uno Application

Here you can find step-by-step tutorials that will give a brief introduction to the features currently available in the Perinno-Uno app and how to use them for a remote collaboration session. Note that Perinno-Uno app is constantly being upgraded so some flow mentioned here may change without notice. If you have any feedback or query, use the contact form at the bottom of the page or send a mail at contact@perinno.com. 

Tutorial I: Connection Establishment between Uno Web App and Perinno HoloLens App

This tutorial will introduce the steps involved for connection establishment between Uno Web App and Perinno HoloLens App. Download, install and launch the Perinno HoloLens App on HoloLens before starting this tutorial.

Step 1: Go to www.perrino.com/uno. Currently, Chrome Version 55 and above is supported. The computer should have audio-video capability for full functionality. The app does system requirements check and if there is something that is missing or unsupported it will show the message as in Fig. 1. Further, the app may have a limited functionality on a mobile device. If the system meets the requirement, no message is displayed.

Figure 1: Test result if the system requirements are not satisfied

Step 2: If the system meets the requirements, login options are shown as in Fig. 2. User can login with any of the available options.

Figure 2: Available login options 

Step 3: Once, login is successful, user will be prompted to create a room or join an existing room as shown in Fig. 3.

Figure 3: Prompt for joining or creating a room

It should be noted that for a collaboration session with HoloLens, it is the Perinno HoloLens App that creates the room ID. When the Perinno HoloLens App is launched on the HoloLens, room ID is automatically created (See Tutorial 1A, below for more details). The remote user having the HoloLens needs to communicate this room ID to the Uno Web App user that should be entered in this prompt box. User may sign out at this stage by clicking on the avatar image on the top right corner of the prompt box.

Step 4: Once the room is joined, a list as shown in Fig. 4 will be displayed.

Figure 4: List showing the HoloLens in the same room

The list will also show the user logged through the Uno Web App. However, it cannot be selected to make a call. The connected HoloLens ID where the Perinno HoloLens App is running can be selected and call can be made. Once the call is made, the application will ask for the permission to use webcam and the microphone as shown in Fig. 5. Allow the access to proceed.

Figure 5: Request for permission to use webcam and microphone.

 Step 5:  Once the permission is granted, a connection is established between the Uno Web App and the Perinno HoloLens App. User can then see two video frame as shown in Fig. 6. One is larger and is the remote video (from HoloLens in this case) and a smaller frame at the left top corner is the Uno Web App user’s local video. The local video frame can be moved and placed at any convenient position on the page.

Figure 6: Remote and local video tag.

At this stage as well, if the user wants to sign out, it can be done by clicking on the avatar at the top right corner of the page and then selecting SIGN OUT in the box as shown in Fig. 7. The box also displays that by which of the login options the user is logged in (if applicable).

Figure 7: Sign out option.

That’s all. Now an audio-video call session starts between the Uno Web App and the Perinno HoloLens App where the call is initiated from the web app. In the next tutorial, other features will be described. Feedback or any questions can be send to us using the form at the end of this page. 

 
Tutorial IA: Making call from Perinno HoloLens App to Uno Web App

In the first tutorial, we saw that how  Uno Web App can make a call to Perinno HoloLens App. It was also discussed that it is the Perinno HoloLens App that creates the room ID. In this tutorial the flow from Perinno HoloLens App to make a call to  Uno Web App  will be discussed.

Step 1: When the Perinno HoloLens App is launched, it will create a room ID as shown in Fig. 1 below. Room ID is the HoloLens device name suffixed by two random digits. Details how to set the HoloLens device name can be found here. This is the room ID that has to be shared with the user running Uno Web App  (see Step 3, Tutorial I).

Figure 1: Initial display screen for Perinno HoloLens App

Step 2: Once the remote user on Uno Web App  has joined the  room, a screen will be displayed displaying the name of the connected user as shown in Fig. 2. The default image on the screen will be replaced by the user’s avatar image (if present). Voice command: Call Remote User can be spoken to call Uno Web App or Perinno HoloLens App will wait for a call from Uno Web App (as discussed above in Tutorial I).

 Figure 2: Display screen after the Uno Web App user has connected

Step 3: If the call is made from Perinno HoloLens App,  at the Uno Web App side, a ringer will go on and the user has option to accept or reject the call as shown in Fig. 3.

Figure 3: Display at the Uno Web App when Perinno HoloLens App has initiated a call.

Step 4: If the call is accepted, Perinno HoloLens App, can see the remote video screen (video from webcam of the device running Uno Web App). The remote video screen is movable and Perinno Hololens App user can drag and place it anywhere in 3D environment by touch tap-and-hold hand gesture. If the Uno Web App, rejects the call, user at the Perinno HoloLens App will be notified by voice that user at the Uno Web App is busy and it will again go to wait state i.e. it will wait for a call or a call can again be tried by the voice command. If the remote user at the Uno Web App disconnects, it will also be notified at the Perinno HoloLens App.

That’s all for this tutorial. Now, we know how to make a call from Perinno HoloLens App to Uno Web App! Feedback or any questions can be send to us using the form at the end of this page.

 
Tutorial II: Introduction to various functionalities in the Uno Web App
This tutorial will give a brief introduction to various buttons and their functionality in the Uno Web App. The buttons are divided into four categories plus one extra button for demo session. Hover the mouse pointer over the group buttons at the footer of the Uno Web App too see the tooltip with the name of the button. Click the button to expand the group buttons to see different buttons within the group. Each button is associated with a functionality. The tooltip will give an idea of the functionality. Note that Uno Web App can even be used from browser to browser (and not necessarily with just the Perinno HoloLens App) with reduced functionalities.

Group 1: Call Button

 

Call button consists of 3 different buttons as shown in the left figure. They are used to control the audio-video or ending the call.

(a) The top button can be used to disable or enable the local video stream. If the local video stream is disabled, the remote user cannot see your video.

(b) Mute or unmute the audio with the middle button.

(c) Hang the on-going call with the remote user using the bottom button.

 

Group 2: 3D Collaboration Button

3D Collaboration button consists of set of three button that is used for 3D collaboration session as shown in the left figure.

(a) Use the top button for sending a local 3D CAD file to Perinno HoloLens App.

(b) Use the middle button to send a 3D CAD file stored in cloud. Current support is for Dropbox.

(c) Bottom button is for manipulating a 3D model sent to the Perinno HoloLens App. Currently, it is only enabled for a 3D demo session. The details of manipulation is described in tutorial III.

 

Group 3: Tools Button

Tools button consists of set of four buttons as shown in the left figure. They incorporate various functionalities as described below.

(a) The top button can be used to scribble over the incoming video stream frame. The scribble will then appear on the remote user side. If remote user is on Perinno HoloLens App, it will appear in the 3D environment. The details will be described in Tutorial IV.

(b) Use the second button to send the image file to the remote user. If the remote user is on Perinno HoloLens App, it will appear in 3D environment and can be moved with touch tap-and-hold hand gesture.

(c) Click the third button to share the screen to the remote user. This require a Chrome extension Screen Capture offered by www.webrtc-experiment.com.

(d) The chat message can be send to the remote user by clicking the bottom button.

Group 4: Record Session Button

Record Session has two buttons for recording the on-going session or capturing the remote video frame.

(a) Use the top button to capture the remote video frame. The captured image will be saved in Download folder in Windows OS.

(b) Click the bottom button to start recording the remote video. Click again to stop the recording and download it to Download folder in Windows OS.

 

Demo Button

The demo button is to start a 3D demo session. Clicking it will initiate a demo session both at Uno Web App as well as at the Perinno HoloLens App. A turbine model will be downloaded at both the Uno Web App and at the Perinno HoloLens App. Manipulate it at the Uno Web App and the changes will be reflected in the Perinno HoloLens App. More about it in a later tutorial.

Congratulations! Now you have a basic understanding of various buttons and associated functionalities  of the Uno Web App. Use the form at the end of this page for sending feedback or queries.

 
Tutorial III: A 3D Demo Session

A simple demo session for 3D object manipulation is described here. User manipulate the 3D model at the Uno Web App and this will get reflected on the 3D model displayed in the Perinno HoloLens App  augmented reality environment. Note that demo session is not an implementation of all the available functionalities, however, it give an idea of what the Perinno-Uno App is capable of doing.

Step 1: Establish an audio-video call between the Uno Web App and the Perinno HoloLens App as described above in Tutorial I or IIA.

Step 2: Click on the 3D Demo button (Fig. 1) to initiate a 3D demo session. Once this button is clicked, on the Uno Web App user will be given next instruction as shown in Fig. 2.

Figure 1: 3D Demo button

Figure 2: Instruction to click next button after clicking 3D Demo button.

On the Perinno HoloLens App side, a voice can be heard saying, “demo session has started, Please wait while 3D model is being loaded”, indicating an on-going demo session. A turbine model will then be displayed in the augmented reality environment of HoloLens as seen in Fig. 3.

Figure 3: Turbine model in 3D augmented reality environment.

Step 3: In this step we will see how a model part at the Uno Web App can be selected and the same part will be then selected at the model displayed in the Perinno HoloLens App environment. This mimics a scenario where the user at the Uno Web App is discussing about a particular part to the remote user at the Perinno HoloLens App.

Click the 3D Collaboration button on the Uno Web App as shown in Fig. 2 above. Then click on the ‘Manipulate 3D Model’ button shown in Fig. 4. Currently, this button is only enabled in the demo session. This will open an Autodesk viewer where the same turbine model will get loaded as seen in Fig. 4.

Figure 4: Manipulate 3D button.

Figure 5: Portal with the turbine model with a selected part shown in blue.

Click on any part of the model to select it. It will turn blue as shown in Fig. 5. Doing this will also select the same part of  the turbine model in Perinno HoloLens App augmented reality environment. This selection can be seen in red color at the Perinno HoloLens App as shown in Fig. 6. The selected part can be moved with touch tap-and-hold gesture at the Perinno HoloLens App user as shown in Fig. 7. Note that while this happens, audio-video session is on-going in the background, though the Perinno HoloLens App environment cannot be seen due to the Autodesk viewer in the foreground. Perinno Hololens App user can still the Uno Web App user’s video (if enabled).

Figure 6: Selected part at the augmented reality environment of Perinno HoloLens App

Figure 7: Selected part moved by touch tap-and-hold gesture at the augmented reality environment of Perinno HoloLens App

Similar to above process, any other part can be selected and moved. More than one part can also be selected.

Step 4: In this step we will see how exploding the model at the Uno Web App to get an exploded view will result in exploded view of the model at the Perinno HoloLens App side. Further, we will deal with selecting and moving a part from the exploded view. This is useful for a scenario where a part may be occluded by other parts and the Uno Web App user is giving information about that occluded part to the Perinno HoloLens App user.

To explode the model, model should be in initial unmodified state. Use command Reset model  to bring the model in its initial unmodified state if it has been modified. On the Uno Web App side, click on the Autodesk Viewer’s explode button. The button is shown in Fig. 8.

Figure 8: The explode model button in the Autodesk Viewer

Once the explode model button is clicked, a slide-bar will appear. Increasing the value on the slide-bar will move the parts away from each other to expose an exploded view. Higher the value, the parts are farther away. The exploded view of turbine can be seen in Fig. 9 on the Uno Web App side.

  Figure 9: The exploded view of the turbine at the Uno Web App side. A selected part can also be seen in blue.

On the Perinno HoloLens App side, clicking on explode button and sliding the slider on Uno Web App will result in the exploded view of the turbine in augmented reality as seen in the Fig. 10. Note that for the demo session, exploded view slider on the Uno Web App has no effect at the Perinno HoloLens App side and the explosion is fixed.

Figure 10: The exploded view of the turbine at the Perinno HoloLens App side. A part selected by Uno Web App user is in red. A part selected by gaze of Perinno HoloLens user is in green.

A part selected by the Uno Web App user as seen in Fig. 9 with blue color in the exploded view will appear in red on the Perinno HoloLens App side. This part is occluded by other parts (in non-exploded view) and hence exploded view makes it easy to visualize. Moreover, this part can be moved by touch tap-and-hold hand gesture. Other parts remain fixed and cannot be moved. However, if user at the Perinno HoloLens App wants to move a particular part that is not selected by the Uno Web App user, it can be done by simply gazing at it. This will turn the color of selected part green and can then be moved by touch tap-and-hold gesture.

Note that Step 3 and Step 4 are not an ordered flow. Any one can be done in any order. The only requirement for an explosion of model to occur at the HoloLens side is that it should be in its unmodified state. 

Congratulations! Now you have learned some basics of 3D collaboration session through this demo tutorial. Feedback and queries can be send using the contact form at the bottom of this page.

 
Tutorial IV: Annotation in 3D augmented reality environment

In this tutorial we will discuss how to annotate in the Perinno HoloLens App augmented reality environment from the Uno Web App. This can be visualized as an use-case where user at the Uno Web App wants to point at an object, or encircle an object of interest in the video from the Perinno HoloLens App.

Step 1: Click on the Scribble button that is the top button in the Tools button set. It is shown in Fig. 1.  Clicking this button will stop the remote video and a static image frame will be displayed on which the Uno Web App user can annotate or scribble.

Figure 1: Scribble button

Step 2: Scribble on the image frame by holding down the left mouse button. Release it to stop scribbling. Press down again if you want to scribble again. You can encircle an object of interest, draw an arrow to point at an object or write something. An example is shown in Fig. 2.

Figure 2: Scribble on a remote video frame at the Uno Web App end.

This will appear on the Perinno HoloLens App augmented reality environment as shown in Fig. 3.

Figure 3: Scribble in the augmented reality environment of the Perinno HoloLens App.

Step 4: To restart the remote video and clear the scribble, click on the Erase Scribble button (Fig. 4). Note that Scribble button toggles to Erase Scribble button after clicking and vice-versa. This will also clear the scribble or the annotation from the Perinno HoloLens App environment. 

Figure 4: Erase Scribble button.

Congratulations! Now you know how to annotate in the Perinno HoloLens App augmented reality environment from the Uno Web App. Use the form at the end of this page for sending feedback or queries.

 Fill and submit the form below for feedback or questions: