Fitzroy logo
Robotics & Physical AI

Train in simulation. Deploy with confidence.

Fitzroy designs domain-specific virtual training environments, develops purpose-built control policies, and builds the complete AWS machine learning pipeline that carries robot learning into physical operations.

Start a ConversationRead the Physical AI Brief
Working stack

MuJoCo PPO GRPO TD3 SAC DQN λ-SARSA PyTorch C++17 AWS

Industrial robotic arm beside its matching virtual simulation environment
Where we help

The full path from a virtual world to working machines.

We connect simulation, learning, validation, edge deployment, and field feedback into one domain-specific engineering system—so teams can iterate faster without treating the real world as the first test environment.

Simulation environment and digital-twin design

Synthetic data and domain scenario generation

Reinforcement and imitation learning pipelines

PPO, GRPO, TD3, SAC, DQN, λ-SARSA, and actor–critic systems

Domain-specific policy and reward design

AWS ML platform architecture and MLOps

Sim-to-real transfer and system calibration

Edge inference and fleet deployment

Safety validation, observability, and iteration

Telemetry-driven evaluation and retraining

When to engage

Where robot learning usually loses momentum.

The hardest problems tend to sit between disciplines: controls and ML, simulation and hardware, research code and production operations. The failure is rarely the algorithm—it is everything around it.

Physical testing is the bottleneck

Hardware access, safety constraints, or slow reset cycles make iteration too expensive to run entirely in the real world.

Policies work only in the demo

A model succeeds in a narrow environment but fails when lighting, payloads, friction, geometry, or operator behavior changes.

Simulation and deployment are separate efforts

Training produces a model artifact, but there is no reliable path into the robot runtime, fleet controls, or release process.

Field failures do not improve the model

Telemetry exists, but it is not structured into scenarios, evaluation sets, and the next training cycle.

The sim-to-real system

One learning loop. Built to survive reality.

The model is only one part of the system. We engineer the environment, training pipeline, transfer tests, runtime, and feedback path together so each field deployment makes the next policy stronger.

Continuous learning architecture
  1. Model

    Build the world

    Recreate the task, sensors, robot dynamics, and operational edge cases in a controllable virtual environment.

  2. Learn

    Train at scale

    Run parallel reinforcement, imitation, or hybrid learning loops without consuming physical fleet time.

  3. Prove

    Test the transfer

    Measure robustness across randomized conditions, hardware limits, safety envelopes, and unseen scenarios.

  4. Operate

    Deploy and observe

    Package policies for the target runtime, instrument field behavior, and route real-world data back into training.

Field data returns to 01 · virtual → physical → virtual
  • Domain randomization
  • Hardware-in-the-loop
  • Policy evaluation
  • Fleet telemetry
Feedback active
Robot learning methods

Choose the learning system around the task.

We select and combine methods based on action space, reward quality, hardware limits, safety constraints, and the amount of real-world data available—not because one algorithm is fashionable.

Domain-specific by design

Factory robotics and household robotics are separate examples—not data sources for one combined model. Each program gets its own environments, reward system, data boundary, policy artifacts, validation gates, and deployment path.

Choose a methodPPO · 01 / 05
01 / 05Training architecture
PPO

Proximal Policy Optimization

A practical default for stable, continuous control: walking, reaching, grasping, carrying, and coordinating whole-body motion across varied simulated environments.

Examples: factory handling OR a separate household-chore policy
Technical drawingPPO loop
02 / 05Training architecture
GRPO

Group Relative Policy Optimization

Compares groups of attempts against one another, making it useful when a task can be completed in several valid ways and outcomes can be ranked with clear reward functions.

Examples: factory task planning OR a separate long-horizon chore policy
Technical drawingGRPO loop
03 / 05Training architecture
DQN

Deep Q-Network

Learns the value of discrete actions. It fits bounded decisions such as select, stop, grasp, place, route, or recover inside a larger humanoid control stack.

Examples: factory recovery logic OR a separate household task selector
Technical drawingDQN loop
04 / 05Training architecture
λ-SARSA

SARSA with Eligibility Traces

Learns on-policy from the sequence the robot actually follows, carrying delayed task rewards backward across recent decisions through eligibility traces.

Examples: production routes OR a separate household routine policy
Technical drawingλ-SARSA loop
05 / 05Training architecture
A–C

Actor–Critic Network

Pairs an actor that selects motion with a critic that estimates expected return—the foundation behind many continuous-control systems for dexterous and mobile robots.

Examples: industrial manipulation OR a separate home-assistance controller
Technical drawingA–C loop
Public work

Rather than describe the capability, here is the code.

FastTD3 is an open-source C++17 implementation of FastTD3—a high-performance variant of Twin Delayed Deep Deterministic Policy Gradient (TD3) built for humanoid control—released under the MIT license.

View FastTD3 on GitHub
Technical drawingFastTD3 · TD3 loop
Twin critics limit overestimation, the actor updates on a delay, and target networks move through soft updates rather than hard copies.
Implementation

Two backends

An Eigen-only build with no LibTorch dependency, and a full LibTorch/PyTorch build. Replay buffer, empirical normalization, target networks with soft updates, and Gaussian exploration noise.

Simulation

MuJoCo integration

A Python–C++ bridge through pybind11 running Humanoid-v5, HumanoidStandup-v5, Walker2d, HalfCheetah, Ant, and Hopper, with observation and reward normalization.

Engineering

Tested and built in CI

Google Test coverage for the replay buffer, normalizer, clipping, TD error, Huber loss, and soft updates, with builds automated through GitHub Actions.

Who does the work

Senior execution, not account management.

Berk Delibalta, Founder & Principal — M.S. Computer Science, AWS Solutions Architect, with enterprise technology experience at Deloitte, IBM, Zurich, and Thales. The person you speak to is the person who writes the code.

FastTD3 is research and educational code, not a product. It is shared as evidence of how the work is done: implemented from the algorithm up, wired to a real physics engine, and tested.

AWS ML platform

We build the whole pipeline around the policy.

From raw sensor data and parallel simulation to training, evaluation, registry, staged edge release, telemetry, and retraining, Fitzroy can own the AWS platform—not just hand over a model file. We isolate model groups, data, evaluation gates, and release channels by operating domain.

  1. 01

    Capture & store

    Isolated domain data, demonstrations, sensor streams, and field telemetry in Amazon S3

  2. 02

    Simulate at scale

    AWS Batch or Amazon EKS workers running parallel GPU environments

  3. 03

    Train & evaluate

    Amazon SageMaker AI or custom PyTorch and Ray training pipelines

  4. 04

    Register & approve

    Separate domain model groups, evaluation gates, Model Registry, and Amazon ECR

  5. 05

    Deploy to robots

    Signed, staged edge releases through AWS IoT Greengrass

  6. 06

    Observe & retrain

    CloudWatch, fleet telemetry, drift signals, and automated retraining

AWS reference architecture

End-to-end robot learning and deployment on AWS.

Multi-account platform mapDeployment flowFeedback + retraining
  1. 01DATA + DEVELOPMENT ACCOUNT, then curated domain data
  2. 02TRAINING + ARTIFACT ACCOUNT, then approved artifact
  3. 03CI/CD ACCOUNT, then promote
  4. 04STAGING ACCOUNT · CANARY ROBOTS, then release gate passed
  5. 05PRODUCTION ACCOUNT · DOMAIN FLEETS, then telemetry · drift · incidents
  6. 06OPERATIONS ACCOUNT, then retrain event → back to 02
RETRAIN EVENT → BACK TO 02
Deliverables

A working system, not a model handoff.

Simulation and training system

A reproducible environment, learning pipeline, reward design, evaluation suite, and policy training stack built around the robot and task.

Complete AWS ML pipeline

The data, GPU training, evaluation, registry, release automation, observability, security, and retraining infrastructure behind the models.

Robot and fleet deployment

A production path for packaging, staged edge releases, monitoring, rollback, and continuous improvement across physical machines or a growing fleet.

How engagements work

Scoped, fixed-price, and small enough to say yes to.

Most programs begin with a feasibility review. Each stage ends with something your team can run—and a clear decision on whether the next stage is worth funding.

  1. 01 · Assess

    Feasibility review

    1–2 weeks

    A written assessment of whether your task suits reinforcement learning at all, what the simulation must capture, and where the sim-to-real risk sits.

    Often ends in a recommendation not to proceed.

  2. 02 · Architect

    Environment build

    3–5 weeks

    A validated simulation environment—typically MuJoCo—for your robot and task, with reward specification, a baseline policy, and a reproducible training setup your team can run.

  3. 03 · Build · Productionize

    Training engagement

    6–12 weeks

    Policy development through evaluated performance across randomized conditions, with the transfer gap measured and reported, the evaluation harness, and handover documentation.

  4. 04 · Support

    Deployment & ongoing

    Retainer

    Edge packaging, staged fleet releases, telemetry-driven retraining, and continued iteration alongside your engineering team—typically two to three days a week.

Pricing is fixed per engagement and quoted after a short scoping call. Each stage follows Fitzroy’s assess, architect, build, productionize, and support method.

See how we work

A defined starting point

Robot-learning feasibility review

Is a simulation-trained policy appropriate for this robot, task, and operating domain?

Typically one to two working weeks; fee, access, deliverables, and acceptance criteria agreed in the statement of work.

Review scope and deliverables →
Inspect public work and reference analyses →Understand the diagnostic methodology →Review security and governance questions →
Start with a feasibility review

Know whether your task is worth training before you build.

One to two weeks, fixed price, and a written answer on whether simulation-trained policies can work for your robot—including when the answer is no.

Book a CallStart a Conversation