Default to SharedStorage

[christiangnrd]

No description provided.

[github-actions]

Metal Benchmarks

Details

Benchmark suite	Current: 2e3d110	Previous: 265ec1d	Ratio
array/accumulate/Float32/1d	407083 ns	406333 ns	1.00
array/accumulate/Float32/dims=1	378292 ns	373000 ns	1.01
array/accumulate/Float32/dims=1L	8812750 ns	8857208 ns	0.99
array/accumulate/Float32/dims=2	440208 ns	456166 ns	0.97
array/accumulate/Float32/dims=2L	2696708 ns	2134875 ns	1.26
array/accumulate/Int64/1d	843084 ns	876417 ns	0.96
array/accumulate/Int64/dims=1	940875 ns	945958 ns	0.99
array/accumulate/Int64/dims=1L	9530542 ns	9637541 ns	0.99
array/accumulate/Int64/dims=2	1248667 ns	1084833 ns	1.15
array/accumulate/Int64/dims=2L	6530542 ns	6745041 ns	0.97
array/broadcast	223959 ns	201959 ns	1.11
array/construct	2042 ns	2125 ns	0.96
array/permutedims/2d	452750 ns	457000 ns	0.99
array/permutedims/3d	707083 ns	859625 ns	0.82
array/permutedims/4d	1170000 ns	1034250 ns	1.13
array/private/copy	233792 ns	249041 ns	0.94
array/private/copyto!/cpu_to_gpu	197334 ns	205667 ns	0.96
array/private/copyto!/gpu_to_cpu	197333 ns	196166 ns	1.01
array/private/copyto!/gpu_to_gpu	198291 ns	172291 ns	1.15
array/private/iteration/findall/bool	1053084 ns	1111750 ns	0.95
array/private/iteration/findall/int	1231959 ns	1244917 ns	0.99
array/private/iteration/findfirst/bool	1109542 ns	1091375 ns	1.02
array/private/iteration/findfirst/int	1121417 ns	1115667 ns	1.01
array/private/iteration/findmin/1d	1227750 ns	1267500 ns	0.97
array/private/iteration/findmin/2d	1073416 ns	910291 ns	1.18
array/private/iteration/logical	1682875 ns	1855042 ns	0.91
array/private/iteration/scalar	1210375 ns	1212750 ns	1.00
array/random/rand/Float32	431500 ns	435500 ns	0.99
array/random/rand/Int64	517584 ns	537375 ns	0.96
array/random/rand!/Float32	394500 ns	370500 ns	1.06
array/random/rand!/Int64	404333 ns	319125 ns	1.27
array/random/randn/Float32	390583 ns	387500 ns	1.01
array/random/randn!/Float32	342250 ns	335375 ns	1.02
array/reductions/mapreduce/Float32/1d	253084 ns	390792 ns	0.65
array/reductions/mapreduce/Float32/dims=1	342750 ns	321541 ns	1.07
array/reductions/mapreduce/Float32/dims=1L	607708 ns	635834 ns	0.96
array/reductions/mapreduce/Float32/dims=2	342375 ns	330083 ns	1.04
array/reductions/mapreduce/Float32/dims=2L	989458 ns	839250 ns	1.18
array/reductions/mapreduce/Int64/1d	438042 ns	582958 ns	0.75
array/reductions/mapreduce/Int64/dims=1	624666 ns	542875 ns	1.15
array/reductions/mapreduce/Int64/dims=1L	1045625 ns	1034958 ns	1.01
array/reductions/mapreduce/Int64/dims=2	780959 ns	645417 ns	1.21
array/reductions/mapreduce/Int64/dims=2L	2173541 ns	2152625 ns	1.01
array/reductions/reduce/Float32/1d	255500 ns	345709 ns	0.74
array/reductions/reduce/Float32/dims=1	336709 ns	319917 ns	1.05
array/reductions/reduce/Float32/dims=1L	630583 ns	636458 ns	0.99
array/reductions/reduce/Float32/dims=2	218917 ns	185833 ns	1.18
array/reductions/reduce/Float32/dims=2L	458334 ns	446292 ns	1.03
array/reductions/reduce/Int64/1d	447458 ns	584041 ns	0.77
array/reductions/reduce/Int64/dims=1	626250 ns	619208 ns	1.01
array/reductions/reduce/Int64/dims=1L	1033709 ns	1039833 ns	0.99
array/reductions/reduce/Int64/dims=2	248375 ns	223500 ns	1.11
array/reductions/reduce/Int64/dims=2L	647584 ns	633625 ns	1.02
array/shared/copy	136458 ns	141875 ns	0.96
array/shared/copyto!/cpu_to_gpu	38041 ns	37208 ns	1.02
array/shared/copyto!/gpu_to_cpu	38916 ns	37667 ns	1.03
array/shared/copyto!/gpu_to_gpu	39584 ns	38208 ns	1.04
array/shared/iteration/findall/bool	1071417 ns	1089042 ns	0.98
array/shared/iteration/findall/int	1234500 ns	1233750 ns	1.00
array/shared/iteration/findfirst/bool	948041 ns	781083 ns	1.21
array/shared/iteration/findfirst/int	969584 ns	804833 ns	1.20
array/shared/iteration/findmin/1d	1095750 ns	1099875 ns	1.00
array/shared/iteration/findmin/2d	1080958 ns	1075000 ns	1.01
array/shared/iteration/logical	1545750 ns	1516917 ns	1.02
array/shared/iteration/scalar	3552 ns	3567.625 ns	1.00
array/sorting/1d	1620334 ns	1834667 ns	0.88
array/sorting/2d	8113125 ns	8329959 ns	0.97
integration/byval/reference	1104791 ns	1100500 ns	1.00
integration/byval/slices=1	1107583 ns	1103209 ns	1.00
integration/byval/slices=2	2021959 ns	2039959 ns	0.99
integration/byval/slices=3	6528917 ns	14253459 ns	0.46
integration/metaldevrt	386084 ns	370166 ns	1.04
kernel/indexing	191458 ns	171875 ns	1.11
kernel/indexing_checked	363458 ns	369416 ns	0.98
kernel/launch	1733.3 ns	1762.5 ns	0.98
kernel/rand	393167 ns	373750 ns	1.05
latency/import	2041007666 ns	2075632833 ns	0.98
latency/precompile	39287833709 ns	39922933042 ns	0.98
latency/ttfp	2384841583 ns	2421210791 ns	0.98
metal/synchronization/context	595.9719101123595 ns	598.15 ns	1.00
metal/synchronization/stream	330.54017857142856 ns	330.8918918918919 ns	1.00

This comment was automatically generated by workflow using github-action-benchmark.

[codecov]

Codecov Report

❌ Patch coverage is 42.85714% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.37%. Comparing base (265ec1d) to head (2e3d110).

Files with missing lines	Patch %	Lines
src/Metal.jl	0.00%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #822      +/-   ##
==========================================
+ Coverage   86.25%   86.37%   +0.12%     
==========================================
  Files          76       76              
  Lines        5296     5300       +4     
==========================================
+ Hits         4568     4578      +10     
+ Misses        728      722       -6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[christiangnrd]

Could the github actions failure be related to the compilation failures?

[maleadt]

I'm not sure how. There's a known issue with ObjC errors leaking outside of their retain/release scope and crashing during error reporting, but that's read-only and shouldn't cause a crash in LLVM.

[maleadt]

I guess the remaining question is semantics. Do we want to allow scalar iteration on shard memory so that it can be used with CPU code? It'll never be as fast as Array, but if we port the dirty memory flag tracking from CUDA.jl we can get it down to a couple of ns (as opposed to ~1ns for an Array getindex or so). If we want to add back scalar iteration checking we add another couple of ns for the TLS check on every access.

[christiangnrd]

i.e. unsafe_wrap(Array, ... wouldn't be needed?

[maleadt]

Right, that's the current design of CUDA.jl:

Precompiling CUDA finished.
  12 dependencies successfully precompiled in 43 seconds. 90 already precompiled.

julia> CUDA.allowscalar(false)

julia> a = cu([1])
1-element CuArray{Int64, 1, CUDACore.DeviceMemory}:
 1

julia> a[]
ERROR: Scalar indexing is disallowed.

julia> b = cu([1]; unified=true)
1-element CuArray{Int64, 1, CUDACore.UnifiedMemory}:
 1

julia> b[]
1

I'm not entirely convinced this is the best option though. It makes sense, but people often use allowscalar for detecting GPU code. We could tell them they have to use private memory for that, but it still makes allowscalar(false) a lie.

[christiangnrd]

This is also how shared storage currently works

[christiangnrd]

But without the dirty flag so there might be some latent race conditions with shared MtlArrays

[maleadt]

This is also how shared storage currently works

Right, but it never was the default. It would break users doing allowscalar(false) to detect GPU functionality execution on the CPU.

And we definitely need the dirty flag to improve performance here. But that can be follow-up work.

[christiangnrd]

What of we add a @warn when they call allowscalar the first time if default storage mode is shared?

[maleadt]

I guess that could work, but the interface is not extensible like that right now.

EDIT: I guess we could implement Metal.allowscalar, have it warn, and then call GPUArrays.allowscalar.

[christiangnrd]

New relevant case: unsupported sorts (by != identity) will silently run on the CPU under SharedStorage.