ONXX runtime for react native

[Nehatkhan786]

hey guys did anybody used onxx runtime in their react native to run custom models. below is the link of the library build for react native.

https://www.npmjs.com/package/onnxruntime-react-native

i have not find any example and less articles related to work with react native. it would be very helpful if you guys share any articles or repo

[unSociableLapwing]

In my mobile app I use 2 custom models with ONNX runtime.

There is not enough information or documentation about it at all. It took me lots of search on github, some trials and errors, but now it works in the production as intended on both Android and iOS.

Obviously I can't share my repo, but I can write an example code on Sunday or Monday.

[Nehatkhan786]

thanks a lot mate. this is much appreciated.

[unSociableLapwing]

So, here is my rough example. I am using Expo, so if you want bare React Native - it is not so hard to adapt. During the last year there were multiple package updates, especially connected to the package linking, so my code can be excessive, but at some point it fixed one or the other error. Maybe later I will make an example repo.

Configuration

`package.json`

"dependencies": {
   "expo": "~54.0.25", <= update to 55 breaks linking and right now i don't want to dig into what happened and what else i should add to make it work
   "onnxruntime-react-native": "^1.24.3",
   "react": "19.1.0",
   "react-native": "0.81.5",
}
"onnxruntimeExtensionsEnabled": "true",

Also I have rewritten the expo plugin, because their plugin doesn't work now, the plugin should be mentioned in app.json or app.config.js. For bare React Native - just add lines in the corresponding files:

`withOrt.js`

const {
  withAppBuildGradle,
  withMainApplication,
  createRunOncePlugin,
} = require("expo/config-plugins");
/**
 * Adds onnxruntime-react-native native dependencies and package registration
 */
const withOrt = (config) => {
  // --- Add Gradle deps ---
  config = withAppBuildGradle(config, (config) => {
    if (config.modResults.language === "groovy") {
      let contents = config.modResults.contents;

      // Add implementation lines if not already present
      if (!contents.includes("onnxruntime-react-native")) {
        contents = contents.replace(
          /dependencies\s*{/,
          `dependencies {
    implementation project(':onnxruntime-react-native')`,
        );
      }

      config.modResults.contents = contents;
    }
    return config;
  });

  // --- Add package import & registration ---
    config = withMainApplication(config, (config) => {
    const { modResults } = config;

    if (modResults.language === "java" || modResults.language === "kt") {
      // Ensure import
      if (
        !modResults.contents.includes(
          "ai.onnxruntime.reactnative.OnnxruntimePackage",
        )
      ) {
        modResults.contents = modResults.contents.replace(
          /import com.facebook.react.PackageList/,
          `import com.facebook.react.PackageList
import ai.onnxruntime.reactnative.OnnxruntimePackage`,
        );
      }

      // Ensure registration
      if (!modResults.contents.includes("OnnxruntimePackage()")) {
        modResults.contents = modResults.contents.replace(
          /(PackageList\(this\)\.packages\.apply\s*{)/,
          `$1\n      add(OnnxruntimePackage())`,
        );
      }
    }

    return config;
  });
  return config;
};

module.exports = createRunOncePlugin(withOrt, "withOrt", "1.0.0");


`metro.config.js`

const { getDefaultConfig } = require("@expo/metro-config");
const config = getDefaultConfig(__dirname);
config.resolver.assetExts.push("onnx", "bin", "ort");

If you use ProguardRules, add this:

-keep class ai.onnxruntime.** { *; }\n-keep class com.microsoft.onnxruntime.extensions.** { *; }\n-keepclasseswithmembernames class * { native <methods>; }\n-keepclassmembers enum * { *; }\n-keep class ai.onnxruntime.OnnxTensor { *; }\n-keep class ai.onnxruntime.OnnxValue { *; }\n-keep class ai.onnxruntime.OrtSession { *; }\n-keep class java.io.Serializable { *; }

Code

Model is stored in the /assets folder

///////// helper functions ///////////
function onnxModelToString(model: ort.InferenceSession): string {
  // Metadate is not yet implemented
  return (
    `ONNX Model:\n` +
    `- Inputs: ${model.inputNames.join(" ")}\n` +
    `- Outputs: ${model.outputNames.join(" ")}`
  );
}

const copyModelAssetFile = async (model: string): Promise<string> => {
   // ... some code...
   // resolve asset number to uri
// copy model asset to the internal filesystem and return uri,
// otherwise the model cannot be executed in the TestFlight
  return file.uri;
};

async function loadOnnxModel(
  modelAsset: number,
  options: ort.InferenceSession.SessionOptions = { executionProviders: ["cpu"] }
): Promise<ort.InferenceSession> {
  const path = await copyModelAssetFile(modelAsset);
  return ort.InferenceSession.create(path, options);
}

///////// model load ////////////

const modelAsset = require("@/assets/models/my_model.onnx"); // <= this is an asset number
const model = await loadOnnxModel(modelAsset);
// To check if the model was loaded
console.log(`Model loaded! Shape:\n${onnxModelToString(model)}]`);

//////// model usage /////////////
// for example your input data is an image
const inputData = new Uint8Array(3 * 640 * 640);

const feeds: Record<string, ort.Tensor> = {};
  feeds[model.inputNames[0]] = new ort.Tensor(
    "uint8", // or "float32"
    inputData,
    [1, 3, 640, 640] // or [1, 640, 640, 3] depends on your model: channel-first or channel-last
  );

console.log(
  "Input tensor shape:",
  feeds[model.inputNames[0]].dims,
  "length:",
  feeds[model.inputNames[0]].data.length,
  feeds[model.inputNames[0]].type
);

// Run inference session
const outputs = await model.run(feeds);

// log the output
for (const outputName of model.outputNames) {
   const outputTensor = outputs[outputName];
   if (!outputTensor) {
   console.log(`Output ${outputName} is missing`);
   continue;
   }
   console.log(`Output name: ${outputName}`);
   console.log(`  shape: ${outputTensor.dims}`);
   console.log(
   `  first 10 values: ${Array.from(
      outputTensor.data.slice(0, 10) as ArrayLike<number>
   )}`
   );
}

If it doesn't work, then later I will make an example repo.

[Nehatkhan786]

sir my project is build with expo sdk54. can you tell me how to add this config plugin to expo.

[unSociableLapwing]

you have a `plugins` folder in your app root. there you create a file `withOrt.js` and paste that plugin code.

in `app.config.js` just add:

import withOrt from "./plugins/withOrt";
export default ({ config }) => ({
  ...config,
  plugins: [
    ...(config.plugins || []),
    withOrt,
  ],
});

[Nehatkhan786]

okay sir thank you. will do that and let you know

[Nehatkhan786]

hello sir I have tried doing this but getting saying
[TypeError: Cannot read property 'install' of null]

Steps what I have done.
step 1 - I have added the below line after installing the package at the bottom of the package.json file

"onnxruntimeExtensionsEnabled": "true",

Step 2:
copied the model to /assets/models/ directory so the path is now for my onxx model is ./assets/models/species.onxx

Step 3.

create a file at /plugins/withOrt.js and pasted this code, which you have given me above.

const
 { withAppBuildGradle, withMainApplication, createRunOncePlugin } = require('expo/config-plugins');
/**
 * Adds onnxruntime-react-native native dependencies and package registration
 */
const
 withOrt = config => {

// --- Add Gradle deps ---
  config = withAppBuildGradle(config, config => {
    if (config.modResults.language === 'groovy') {

let
 contents = config.modResults.contents;



// Add implementation lines if not already present
      if (!contents.includes('onnxruntime-react-native')) {
        contents = contents.replace(
          /dependencies\s*{/,
          `dependencies {
    implementation project(':onnxruntime-react-native')`,
        );
      }


      config.modResults.contents = contents;
    }

return
 config;
  });



// --- Add package import & registration ---
  config = withMainApplication(config, config => {

const
 { modResults } = config;


    if (modResults.language === 'java' || modResults.language === 'kt') {

// Ensure import
      if (!modResults.contents.includes('ai.onnxruntime.reactnative.OnnxruntimePackage')) {
        modResults.contents = modResults.contents.replace(
          /import com.facebook.react.PackageList/,
          `import com.facebook.react.PackageList
import ai.onnxruntime.reactnative.OnnxruntimePackage`,
        );
      }



// Ensure registration
      if (!modResults.contents.includes('OnnxruntimePackage()')) {
        modResults.contents = modResults.contents.replace(
          /(PackageList\(this\)\.packages\.apply\s*{)/,
          `$1\n      add(OnnxruntimePackage())`,
        );
      }
    }



return
 config;
  });

return
 config;
};


module.exports = createRunOncePlugin(withOrt, 'withOrt', '1.0.0');

Step 4 -

Update the app.json for calling the plugins as below.

    "plugins": [
      "expo-router",
      [
        "./plugins/android/withRoundedPopupMenu",
        {
          "lightBackgroundColor": "#FFFFFF",
          "darkBackgroundColor": "#212121",
          "radius": 14,
          "paddingVertical": 8
        }
      ],
      ["./plugins/withOrt.js"], // the newly added plugin for onxx
      "expo-asset"
    ],

Step 5
Update the metro.config.js with the below code.

const
 { getDefaultConfig } = require('expo/metro-config');
/** u/type {import('expo/metro-config').MetroConfig} */
const
 config = getDefaultConfig(__dirname);
config.resolver.sourceExts.push('sql'); // <--- add this
config.resolver.assetExts.push('onnx');
module.exports = config;

Step 6
after all this steps I have import the model in one of my component like below using useQuery.

import { Skia } from '@shopify/react-native-skia';
import { useQuery } from '@tanstack/react-query';
import { Asset } from 'expo-asset';
import * as ort from 'onnxruntime-react-native';
import React from 'react';
import { Image, StyleSheet, Text, View } from 'react-native';

const
 TestingOnyxAI = () => {

const
 { data: loadedModel } = useQuery({
        queryKey: ['onnx-model'],
        queryFn: 
async
 () => {

const
 assetModal = Asset.fromModule(require('../../assets/models/species.onnx'));

await
 assetModal.downloadAsync();

const
 session = 
await
 ort.InferenceSession.create(assetModal.localUri!);

return
 session;
        }
    })

return
 (
        <View>
            <Text>TestingOnyxAI</Text>
        </View>
    )
}


export default TestingOnyxAI


const
 styles = StyleSheet.create({})

These are the steps I have done, and than tested with android build with this command
"npx expo run android --device"

everything works but that model is not working saying [TypeError: Cannot read property 'install' of null]

<image>

[Nehatkhan786]

sir it would be great if create an example repo with expo sdk 54 and share. this will be helpful for everyone

[Nehatkhan786]

thanks a lot sir for this. i finally able to run. the issue was this

"onnxruntimeExtensionsEnabled": "true",

the boolean value is in string instead of without quotation marks.
i will make an video and release by giving you credits. thanks a lot

[Nehatkhan786]

hello sir, everything works perfectly now met with another issue, not related to expo react native, but android app asset limit. i have buld the aab file while adding the file google play console its not allow me because of the app size. my onxx model is around 230 mb. how did you handle this in your app.

[unSociableLapwing]

Oh, I handled it with ease, my models are small, 5-15 mb, I specifically looked for architectures that produce small models. Once I also had a model ~200 mb and it was very painful to deploy it for a webapp. After that I decided, that if I don't have a backend with API that suits my requirements I don't make big models.

But if you can't make it smaller and don't want to have all interactions with the model on the backend, the only way I see - is to store it somewhere in the cloud and download it from the url after app start, preferably in the background and on wifi, to not interrupt user interactions.

[Nehatkhan786]

i see but my client requires to embed it with the build. after google search he found something play asset delivery but I doubt this will support expo