Appendices

Supporting documentation and additional resources for the Portalt project

User Manual

Admin User Guide

1. Login to the Admin Dashboard

Use your credentials to access the admin dashboard.

Admin Login Screen
2. Create an Organization

After logging in, you can create a new organization by clicking the "Create Organization" button.

Create Organization Screen
3. Add or Invite Members

Invite team members to your organization by adding their email addresses.

Invite Members Screen
4. Select Your Organization

Choose your organization from the list to access its dashboard.

Organization Selection

1. Create a New Activity

From your organization's home page, click the "Create Activity" button to start a new activity.

Create Activity Screen
2. Configure Activity Settings
Create Activity Screen

Fill in the following details:

  • Activity title
  • Banner image URL (optional)
  • Select VR as the platform
  • Choose the target headset

Click "Create Activity" to finalize.

1. Activity Page Overview

After creating an activity, you'll be directed to the activity page with the scene editor.

Activity Page
2. Adding Objects to Your Scene

To add an object to your scene:

  1. Click the "Add Artifact" button
  2. Select an object file from your computer
  3. The object will be added to your scene
Adding Objects

1. Activating RAG

Navigate to the AI Guide page and activate the RAG (Retrieval-Augmented Generation) feature.

RAG Activation
2. Adding Documents to RAG

After activating RAG:

  1. Click "Add Document to RAG"
  2. Select a PDF file from your computer
  3. The document will be processed and added to the knowledge base

1. Generating Access Codes

To access your scene from the admin executable:

  1. Go to the Activity page
  2. Click "Join Activity" to generate an activity code (valid for 24 hours)
  3. Click "Pair Editor App" to generate a pair activity code
Access Codes
Access Codes
2. Entering Codes in the Viewer

Enter the pair code when configuring the activity in the admin editor scene

Enter the activity join code when joining an activity in the viewer scene

End User Guide

1. Connecting to an Activity

When you launch the Viewer Scene, you'll be prompted to enter connection details:

  • IP Address and Port: Enter the IP address and port provided by your administrator. It's important to use only the IP suggested in the admin dashboard.
  • Activity Join Code: Enter the activity join code provided by your administrator.
Configuration Menu
Important: Always use the IP address and port displayed in the admin dashboard, as shown below:
IP and Port in Dashboard
2. Loading the Scene

After entering the correct connection details, the scene will dynamically pair with the administrator's created activity and load all assets that were placed in the environment.

Scene Loading
3. Interacting in the Environment

Once in the environment, you can:

  • Interact with the RAG Chatbot: Ask questions about the environment and content through voice commands. The AI guide will respond based on the documents uploaded by the administrator.
  • Communicate with Other Users: Interact with other people who have joined the same room through voice chat.
  • Use Gestures: Communicate non-verbally using hand gestures and movements.
  • Explore: Move around the environment and examine the objects placed by the administrator.
Chatbot Interaction
4. Best Practices
  • Ensure your microphone is properly configured for voice interaction.
  • Stay within recommended boundaries of the virtual environment.
  • Follow any specific instructions provided by the activity administrator.
  • For best performance, use a stable internet connection.
  • Report any technical issues to the administrator.

Editor Scene Setup

1. Launching the Editor

The Editor Scene is accessible on PC. Locate and run the Editor executable file to begin.

Editor Executable Icon
2. Loading Screen

When you first launch the editor, you'll be presented with the loading screen.

Editor Loading Screen
3. Help Menu

Click on the Help button in the GUI to access navigation and interaction controls.

Help Button Location

The Help Menu provides instructions on how to navigate and interact with objects in the environment:

Help Menu
Basic Controls:
  • Movement: WASD keys to move forward, backward, and side-to-side
  • Look around: Move the mouse to change your view direction
  • Select objects: Left-click on an object to select it
  • Place objects: Select an object from the menu and click to place
  • Scale objects: Hold Shift + mouse wheel to increase or decrease size
  • Rotate objects: Hold R + mouse movement to rotate selected objects
4. Connecting to Your Activity

To load your assets from the Portalt dashboard:

  1. Enter the pairing code generated from your activity on the Portalt dashboard
  2. Press the Connect button
Pairing Code Entry
5. Working with Objects

After connecting, your assets will be dynamically loaded from the dashboard activity.

Dynamic Objects Loading

You can now:

  • Select objects by clicking on them
  • Place objects by selecting them and clicking where you want them
  • Adjust their position by dragging
  • Scale objects using Shift + mouse wheel
  • Rotate objects using R + mouse movement
6. Saving Your Scene

Once you've arranged your objects as desired, press the Save button in the GUI bar to save your scene configuration. This will update your activity on the Portalt dashboard with the current layout.

Deployment Manual

1. Admin App Requirements
  • Hardware:
    • Minimum 4 CPU cores
    • 16GB RAM
    • 100GB SSD storage
    • High-speed internet connection
  • Software:
    • Node.js 18+
    • Next.js 14
    • Electron
    • TypeScript
    • Docker and Kubernetes tools
    • IBM Cloud CLI
2. Ubiq-Genie Server Requirements
  • Hardware:
    • Minimum 4 CPU cores
    • 16GB RAM
    • 100GB SSD storage
    • High-speed internet connection
  • Software:
    • Python 3.10+
    • Node.js 20+
    • FAISS vector store
    • Sentence-transformers
    • Docker and Kubernetes tools
    • IBM Cloud CLI
3. VR Scenes Requirements
  • Hardware:
    • VR-capable GPU (NVIDIA RTX 3060 or better)
    • 16GB RAM
    • 100GB SSD storage
    • High-speed internet connection
  • Software:
    • Unity 2022.3 LTS or later
    • Unity Hub
    • Android Build Support
    • iOS Build Support
    • Universal Windows Platform Build Support

1. Admin App Installation
  1. Clone the repository:
    git clone https://github.com/SysEngTeam20/portalt-admin-app.git
    cd admin-app
  2. Install dependencies:
    npm install
  3. Set up environment variables:
    cp .env.example .env.local
  4. Configure required services:
    • Set up Clerk Authentication
    • Configure IBM Cloud Object Storage
    • Set up database (SQLite for development, MongoDB for production)
2. Ubiq-Genie Server Installation
  1. Clone the repository:
    git clone https://github.com/SysEngTeam20/rag-enabled-ubiq-genie.git
    cd ubiq-genie
  2. Set up Python environment:
    python -m venv venv
    source venv/bin/activate  # On Windows: venv\Scripts\activate
    pip install -r requirements.txt
  3. Set up Node.js environment:
    cd Node
    npm install
  4. Configure environment variables:
    cp .env.template .env.local
3. VR Scenes Installation
  1. Clone the repository:
    git clone https://github.com/SysEngTeam20/unity-vr-scenes.git
  2. Install Unity Hub and required Unity version (6000.0.25f1 or later)
  3. Open the Unity project in Unity Hub
  4. Install required Unity modules:
    • Android Build Support
    • iOS Build Support
    • Universal Windows Platform Build Support

1. Admin App Configuration

Create and configure the environment file (.env.local):

# Clerk Authentication
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=your_clerk_publishable_key
CLERK_SECRET_KEY=your_clerk_secret_key

# Database Configuration
SQLITE_DB_PATH=./data/portalt.db  # Development
MONGODB_URI=your_mongodb_connection_string  # Production

# IBM Cloud Object Storage
COS_ENDPOINT=your_cos_endpoint
COS_ACCESS_KEY_ID=your_cos_access_key_id
COS_SECRET_ACCESS_KEY=your_cos_secret_access_key
COS_BUCKET_NAME=your_bucket_name
COS_INSTANCE_CRN=your_instance_crn
IBM_CLOUD_REGION=your_region

# API Security
API_SECRET_KEY=your_api_secret_key
2. Ubiq-Genie Server Configuration

Create and configure the environment file (.env.local):

# API Configuration
API_BASE_URL=http://localhost:3000
API_SECRET_KEY=your_jwt_signing_secret

# LLM Configuration
LLM_SERVER=http://localhost:8080
LLM_PORT=8080

# Activity Configuration
ACTIVITY_ID=your_activity_id

# WebSocket Configuration
WEBSOCKET_SERVER_URL=ws://localhost:5001
3. VR Scenes Configuration
  1. Configure build settings in Unity Editor
  2. Set up platform-specific settings
  3. Configure input settings for VR
  4. Set up quality settings for optimal performance

1. Docker Deployment
Admin App
  1. Build Docker image:
    docker build -t portalt-admin:latest .
  2. Test container locally:
    docker run -p 3000:3000 --env-file .env.local portalt-admin:latest
Ubiq-Genie Server
  1. Build Docker image:
    docker build -t ubiq-genie:latest .
  2. Test container locally:
    docker run -p 8000:8000 --env-file .env.local ubiq-genie:latest
2. Kubernetes Deployment on IBM Cloud
Admin App Deployment
  1. Create container registry namespace:
    ibmcloud cr namespace-add portalt
  2. Build and tag Docker image:
    docker build -t portalt-admin:latest .
    docker tag portalt-admin:latest us.icr.io/portalt/portalt-admin:latest
  3. Log in to IBM Container Registry:
    ibmcloud cr login
  4. Push image to registry:
    docker push us.icr.io/portalt/portalt-admin:latest
  5. Create secrets:
    kubectl create secret generic admin-app-secrets \
      --namespace portalt \
      --from-literal=clerk-publishable-key=$NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY \
      --from-literal=clerk-secret-key=$CLERK_SECRET_KEY \
      --from-literal=cos-endpoint=$COS_ENDPOINT \
      --from-literal=cos-access-key-id=$COS_ACCESS_KEY_ID \
      --from-literal=cos-secret-access-key=$COS_SECRET_ACCESS_KEY \
      --from-literal=cos-bucket-name=$COS_BUCKET_NAME \
      --from-literal=cos-instance-crn=$COS_INSTANCE_CRN \
      --from-literal=ibm-cloud-region=$IBM_CLOUD_REGION \
      --from-literal=api-secret-key=$API_SECRET_KEY
  6. Deploy application:
    kubectl apply -f k8s/admin-app/
Ubiq-Genie Server Deployment
  1. Build and tag Docker image:
    docker build -t ubiq-genie:latest .
    docker tag ubiq-genie:latest us.icr.io/portalt/ubiq-genie:latest
  2. Push image to registry:
    docker push us.icr.io/portalt/ubiq-genie:latest
  3. Create secrets:
    kubectl create secret generic ubiq-genie-secrets \
      --namespace portalt \
      --from-literal=api-base-url=$API_BASE_URL \
      --from-literal=api-secret-key=$API_SECRET_KEY \
      --from-literal=llm-server=$LLM_SERVER \
      --from-literal=llm-port=$LLM_PORT \
      --from-literal=activity-id=$ACTIVITY_ID \
      --from-literal=websocket-url=$WEBSOCKET_SERVER_URL
  4. Deploy application:
    kubectl apply -f k8s/ubiq-genie/
VR Scenes Deployment
  1. Build the application in Unity:
    • Select target platform (Android/iOS/Windows)
    • Configure build settings
    • Generate build
  2. Deploy to target platforms:
    • Android: Generate APK/Bundle and publish to Google Play Store
    • iOS: Generate Xcode project and publish to App Store
    • Windows: Package and distribute through appropriate channels
3. Post-Deployment Verification
Admin App Verification
  1. Check pod status:
    kubectl get pods -n portalt -l app=portalt-admin
  2. Check logs:
    kubectl logs -f deployment/portalt-admin -n portalt
Ubiq-Genie Server Verification
  1. Check pod status:
    kubectl get pods -n portalt -l app=ubiq-genie
  2. Check logs:
    kubectl logs -f deployment/ubiq-genie -n portalt
VR Scenes Verification
  1. Test builds on target devices
  2. Verify VR functionality
  3. Check performance metrics

1. Admin App Issues
  • Database Issues:
    • Check database integrity
    • Verify database permissions
    • Check for database locks
  • File Upload Failures:
    • Validate file size limits
    • Check storage permissions
    • Verify IBM Cloud Object Storage credentials
  • Electron-specific Issues:
    • Check main process logs
    • Verify preload script configuration
    • Check window management
2. Ubiq-Genie Server Issues
  • RAG Integration Issues:
    • Verify token generation
    • Check document access permissions
    • Validate API endpoints
    • Check LLM server connectivity
  • WebSocket Issues:
    • Check WebSocket server status
    • Verify connection settings
    • Check network firewall rules
  • LLM Server Issues:
    • Verify LLM server is running
    • Check model loading status
    • Monitor resource usage
3. VR Scenes Issues
  • Build Issues:
    • Check Unity build settings
    • Verify platform-specific requirements
    • Check for missing dependencies
  • Performance Issues:
    • Check frame rate
    • Monitor memory usage
    • Verify asset optimization
  • VR-specific Issues:
    • Check VR input settings
    • Verify headset compatibility
    • Check tracking system

Development Blog

The blog contains in-depth technical discussions, design decisions, meeting notes, and progress updates that complement the information available on this website.

Monthly Videos

February / March 2025 Update

In this update, we show the integration of the RAG system with the VR environment and demonstrate multi-user interactions.

January 2025 Update

This update covers our progress on developing the admin dashboard interface and the VR scene editing capabilities.

December 2024 Update

In this update, we demonstrate our initial VR environment prototypes and discuss the architecture of our RAG-enabled AI guide system.

November 2024 Update

This update shows our progress on the system design, initial technology stack decisions, and early prototypes of the admin dashboard.

October 2024 Update

Our first monthly update covering project introduction, team structure, initial requirements gathering, and research direction.