Image search

This page covers the details of integrating visual search. Please see this article in the Help Center to learn how visual search works and its functionalities.

Images are attached to the search request payload and uploaded directly to the Find API, via an HTTP Multipart Body.

---

HTTP

POST https://api.partium.io/1/find/search
Authorization: Bearer <<Your Access Token or API Key>>
Content-Type: multipart/form-data; boundary=549844d5-2a2e-4006-8ba9-6ca3423e3fa0
Content-Length: 842545

--549844d5-2a2e-4006-8ba9-6ca3423e3fa0
Content-Disposition: form-data; name="input"
Content-Type: application/json

{
  "language": "en",
  "matches": {
    "organization": "<<Your organization name>>"
  }
}
--549844d5-2a2e-4006-8ba9-6ca3423e3fa0
Content-Disposition: form-data; name="image";
Content-Type: application/octet-stream

<<your image binary content>>
--549844d5-2a2e-4006-8ba9-6ca3423e3fa0--

Python

response = requests.post(
  'https://api.partium.io/1/find/search',
  headers={
    'Authorization': 'Bearer <<Your Access Token or API Key>>',
  },
  files={
    'image': ('image', image_binary_content),
  },
  data={
    'input': json.dumps({
      'language': 'en',
      'matches': {
        'organization': '<<Your organization name>>',
      },
    }),
  },
)

JavaScript

let data = new FormData();
data.append('input', JSON.stringify({
  language: 'en',
  matches: {
    organization: '<<Your organization name>>',
  },
}));
data.append('image', new Blob(image_binary_content, {type: 'application/octet-stream'}));

fetch('https://api.partium.io/1/find/search', {
  method: 'POST',
  headers: {
    Authorization: 'Bearer <<Your Access Token or API Key>>',
  },
  body: data,
}).then(res => {
  ...
});

Partium SDK

Partium.search.performSearch(
  {
    organizationName: '<<Your organization name>>',
    searchLanguage: 'en',
    searchImage: new JsPFile( imageBlob ),
  }
);

note

Replace <<Your Organization Name>> with the provided organization name.

Replace <<Your Access Token or API Key>> with the preferred authentication method. See Authentication.

---

Limits & Constraints

Due to the nature of image search, an asset needs to be transferred from the client to the server. Since images can be large assets, depending on factors such as camera quality, image format, and dimensions, among others, this transfer may significantly impact performance, especially on slow connections.

The techniques employed by Partium during the search do not necessitate high-definition images. Additionally, our tests have demonstrated that there is no tangible benefit to using larger images.

Therefore, we have implemented a limit on the request payload to discourage searches with non-optimized setups, which could result in poor performance and latency. Please see Limits & Constraints for more information.

Maximum file size

The maximum payload size for any request is set at 5.5 MiB. It includes both the image and the search input body payload.

HTTP 413 - Request Entity Too Large is returned whenever this value is exceeded.

The Content-Length header is mandatory for every search request.

HTTP 411 - Length Required is returned in case the header is missing or not properly formatted.

Maximum dimension

There's no requirement for the aspect ratio or format of the image, so it can be a square, a vertical rectangle, or a horizontal rectangle. The limit is applied to the smallest side only.

A limit of 960px is imposed, as our tests have shown that exceeding this size penalizes the transfer latency without adding significant information to the search itself.

Examples: 960x5096, 961x960 and 5096x960 are valid formats while 961x961 and 961x5096 would be rejected.

We do not recommend going above the limit on any of the image sides.

HTTP 422 - Unprocessable Entity is returned in case the limit is extrapolated.

Accepted media types

Supported media types include, but are not limited to: image/png, image/jpeg, image/x-icon, image/icns, image/jp2, and image/tiff.

HTTP 415 - Unsupported Media Type is returned in case an unrecognized media type is received.

Reusing images or continuing image searches

As uploading images is a crucial aspect affecting latency during image searches, all uploaded images receive a unique identifier. This allows them to be reused for future searches without the need for re-uploading.

It's especially useful when presenting filters to the user alongside search results to refine the search. Check Smart Filters for an easy way to achieve this.

The unique id for every image is included in the search results for every image search request as the imageId field. It can then be sent back in a new search request as the reusableImageId field.

---

HTTP

> POST https://api.partium.io/1/find/search
> Authorization: Bearer <<Your Access Token or API Key>>
> Content-Type: multipart/form-data; boundary=549844d5-2a2e-4006-8ba9-6ca3423e3fa0
> Content-Length: 842545
>
> --549844d5-2a2e-4006-8ba9-6ca3423e3fa0
> Content-Disposition: form-data; name="input"
> Content-Type: application/json
>
> {
>   "language": "en",
>   "matches": {
>     "organization": "<<Your organization name>>"
>   }
> }
> --549844d5-2a2e-4006-8ba9-6ca3423e3fa0
> Content-Disposition: form-data; name="image";
> Content-Type: application/octet-stream
>
> <<your image binary content>>
> --549844d5-2a2e-4006-8ba9-6ca3423e3fa0--
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 243
<
< {
<   "sessionId": "cfaac7fc-8c0a-420a-b6ee-f59396861753",
<   "imageId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
<   "results": [
<     {
<       "id": "PM19239.AB"
<     },
<     {
<       "id": "PM19241.AD"
<     },
<     {
<       "id": "PM19239.AF"
<     }
<   ]
< }

...

> POST https://api.k8s.int.partium.io/1/find/search
> Authorization: Bearer <<Your Access Token or API Key>>
> Content-Type: application/json
> Content-Length: 188
>
> {
>   "language": "en",
>   "reusableImageId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
>   "matches": {
>     "organization": "<<Your organization name>>",
>     "text": "automatic dog feeder"
>   }
> }
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 243
<
< {
<   "sessionId": "cfaac7fc-8c0a-420a-b6ee-f59396861753",
<   "imageId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
<   "results": [
<     {
<       "id": "PM19239.AD"
<     },
<     {
<       "id": "PM19241.AB"
<     },
<     {
<       "id": "PM19239.AF"
<     }
<   ]
< }

Python

response = requests.post(
  'https://api.partium.io/1/find/search',
  headers={
    'Authorization': 'Bearer <<Your Access Token or API Key>>',
  },
  files={
    'image': ('image', image_binary_content),
  },
  data={
    'input': json.dumps({
      'language': 'en',
      'matches': {
        'organization': '<<Your organization name>>',
      },
    }),
  },
)
response_body = response.json()
current_image_id = response_body['imageId']

...

response = requests.post(
  'https://api.partium.io/1/find/search',
  headers={
    'Authorization': 'Bearer <<Your Access Token or API Key>>',
  },
  json={
    'language': 'en',
    'reusableImageId': current_image_id,
    'matches': {
      'organization': '<<Your organization name>>',
      'text': 'automatic dog feeder'
    },
  },
)

JavaScript

let data = new FormData();
data.append('input', JSON.stringify({
  language: 'en',
  matches: {
    organization: '<<Your organization name>>',
  },
}));
data.append('image', new Blob(image_binary_content, {type: 'application/octet-stream'}));

let currentImageId = null;

fetch('https://api.partium.io/1/find/search', {
  method: 'POST',
  headers: {
    Authorization: 'Bearer <<Your Access Token or API Key>>',
  },
  body: data,
}).then(response => response.json())
.then(data => {
  currentImageId = data.imageId;
});

...

fetch('https://api.partium.io/1/find/search', {
  method: 'POST',
  headers: {
    Authorization: 'Bearer <<Your Access Token or API Key>>',
  },
  body: {
    language: 'en',
    reusableImageId: currentImageId,
    matches: {
      organization: '<<Your organization name>>',
      text: 'automatic dog feeder'
    },
  },
}).then(response => response.json())
...

note

Replace <<Your Organization Name>> with the provided organization name.

Replace <<Your Access Token or API Key>> with the preferred authentication method. See Authentication.

---

Applying an area of interest

Partium Semantic Search features are capable of extracting information from images to enhance the results. To maximize their capabilities, it's best to minimize noise in the image. Noise refers to any unwanted information apart from the desired part, such as surrounding objects, machines, or background clutter like a table.

To assist the algorithm in focusing on the correct part, users can specify a specific area of the image. This can be achieved by allowing users to draw a rectangle around the desired part, capturing the coordinates of the four corner points, and then sending this information together in the imageOptions.areaOfInterest.

The imageOptions.areaOfInterest is an object representing the x1, y1, x2, y2 fields. x1/y1, x2/y2 represent the top left and bottom right coordinates of the area, expressed as percentages where 0 corresponds to 0% and 1 corresponds to 100%. For example, in the following image, the area of interest can be defined as follows: {"x1": 0.25, "y1": 0.2, "x2": 0.8, "y2": 0.7}:

---

HTTP

> POST https://api.partium.io/1/find/search
> Authorization: Bearer <<Your Access Token or API Key>>
> Content-Type: multipart/form-data; boundary=549844d5-2a2e-4006-8ba9-6ca3423e3fa0
> Content-Length: 842545
>
> --549844d5-2a2e-4006-8ba9-6ca3423e3fa0
> Content-Disposition: form-data; name="input"
> Content-Type: application/json
>
> {
>   "language": "en",
>   "matches": {
>     "organization": "<<Your organization name>>"
>   },
>   "imageOptions": {
>     "areaOfInterest": {"x1": 0.25, "x2": 0.75, "y1": 0.25, "y2": 0.75}
>   }
> }
> --549844d5-2a2e-4006-8ba9-6ca3423e3fa0
> Content-Disposition: form-data; name="image";
> Content-Type: application/octet-stream
>
> <<your image binary content>>
> --549844d5-2a2e-4006-8ba9-6ca3423e3fa0--
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 243
<
< {
<   "sessionId": "cfaac7fc-8c0a-420a-b6ee-f59396861753",
<   "imageId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
<   "results": [
<     {
<       "id": "PM19239.AB"
<     },
<     {
<       "id": "PM19241.AD"
<     },
<     {
<       "id": "PM19239.AF"
<     }
<   ]
< }

Python

response = requests.post(
  'https://api.partium.io/1/find/search',
  headers={
    'Authorization': 'Bearer <<Your Access Token or API Key>>',
  },
  files={
    'image': ('image', image_binary_content),
  },
  data={
    'input': json.dumps({
      'language': 'en',
      'matches': {
        'organization': '<<Your organization name>>',
      },
      'imageOptions': {
        'areaOfInterest': {'x1': 0.25, 'x2': 0.75, 'y1': 0.25, 'y2': 0,75},
      },
    }),
  },
)

JavaScript

let data = new FormData();
data.append('input', JSON.stringify({
  language: 'en',
  matches: {
    organization: '<<Your organization name>>',
  },
  imageOptions: {
    areaOfInterest: {x1: 0.25, x2: 0.25, y1: 0.75, y2: 0.75},
  },
}));
data.append('image', new Blob(image_binary_content, {type: 'application/octet-stream'}));

let currentImageId = null;

fetch('https://api.partium.io/1/find/search', {
  method: 'POST',
  headers: {
    Authorization: 'Bearer <<Your Access Token or API Key>>',
  },
  body: data,
}).then(response => response.json())
.then(data => {
  currentImageId = data.imageId;
});

note

Replace <<Your Organization Name>> with the provided organization name.

Replace <<Your Access Token or API Key>> with the preferred authentication method. See Authentication.

---