Missing includes for faiss when building for Android x86-64

[DavidVentura]

Bug description

faiss does not build on x86-64

How to reproduce

Include marian-dev in a CMake project (in my case, Android) and hit build.

You will get

error: unknown type name '__m128'
  148 | static inline __m128 masked_read(int d, const float *x)
      |               ^

This patch solves the issue

diff --git a/src/3rd_party/faiss/VectorTransform.h b/src/3rd_party/faiss/VectorTransform.h
index f0ed1440..ab34bc93 100644
--- a/src/3rd_party/faiss/VectorTransform.h
+++ b/src/3rd_party/faiss/VectorTransform.h
@@ -21,6 +21,12 @@
 #if defined(__APPLE__) && !defined(__arm64__)
 #include <x86intrin.h>
 #endif
+#if defined(__x86_64__) || defined(_M_X64)
+#include <xmmintrin.h>
+#include <emmintrin.h>
+#include <pmmintrin.h>
+#endif
+

Also reported upstream

Maybe masked_read should be gated by #ifdef __AVX__ instead of #ifdef __SSE__ ?

[onestardao]

yep — this is one of those cross-compilation traps that pops up in FAISS pretty often.
the root problem isn't just missing includes, it's the lack of robust platform guards for AVX/SSE intrinsics.

we catalogued this in a diagnostic set for faiss + embedded targets (android, wasm, etc).
there’s a tested fix strategy for guarding _m256 and masked_read across GCC/Clang variations — happy to share if you're looking for a more generalizable patch structure.

(also: props for surfacing this upstream, most folks just patch & move on.)

[DavidVentura]

happy to share if you're looking for a more generalizable patch structure.

Please do! I had to patch multiple projects following the dependency tree in bergamot-nmt for my project

[onestardao]

awesome — yeah here’s the patch strategy we use for this AVX/SSE platform trap:
👉 https://github.com/onestardao/WFGY/blob/main/ProblemMap/vectorstore-metrics-and-faiss-pitfalls.md

it’s part of a broader diagnostic set we maintain for embedded FAISS issues (esp android builds).
includes both platform guards + semantic layer fallback when patching isn’t enough.

should slot in clean for m256 + masked_read edge cases across GCC/Clang.
LMK if you hit anything weird in bergamot’s tree — we’ve mapped some of those too.

Problem map home : https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md

MIT License , enjoy :)