Context
PR #384 inserts bufferization::createEmptyTensorToAllocTensorPass() immediately before the pre-bufferize cse in buildOnnxToHipPipelineTail, to stop CSE from coalescing the tensor.empty destinations of distinct HIP DPS ops into one SSA value (which led to shared output buffers / overwrites after bufferization — see the detailed write-up: #384 (comment)).
Concern (raised by @fhanuman)
Converting tensor.empty → bufferization.alloc_tensor forces a distinct allocation per op and disables one-shot-bufferize's empty-tensor-elimination. That could increase the buffer count / GPU pool size, i.e. partially prevent buffer optimization. The downstream --hip-pool-allocs still does liveness-based packing, so the final pool may be unchanged — but this needs to be measured on real models.
Task
Compare, with vs without the empty-tensor-to-alloc-tensor pass, on:
Metrics:
hipdnn.pool_size — final GPU pool bytes stamped by --hip-pool-allocs (most direct).- post-bufferize
memref.alloc count.
Method
Dump per-pass IR (HIPDNN_EP_IR_DUMP_PATH + HIPDNN_EP_IR_DUMP_TREE=1) for each model on a build with the pass and a build without it; diff the two metrics.
Outcome
Quantify the delta. If the pool/buffer growth is material, consider a more surgical alternative (e.g. only convert empties that feed a DPS outs, or re-materialize distinct destinations after CSE) instead of converting all empties.
Refs: PR #384, comment #384 (comment)
Context
PR #384 inserts
bufferization::createEmptyTensorToAllocTensorPass()immediately before the pre-bufferizecseinbuildOnnxToHipPipelineTail, to stop CSE from coalescing thetensor.emptydestinations of distinct HIP DPS ops into one SSA value (which led to shared output buffers / overwrites after bufferization — see the detailed write-up: #384 (comment)).Concern (raised by @fhanuman)
Converting
tensor.empty→bufferization.alloc_tensorforces a distinct allocation per op and disables one-shot-bufferize's empty-tensor-elimination. That could increase the buffer count / GPU pool size, i.e. partially prevent buffer optimization. The downstream--hip-pool-allocsstill does liveness-based packing, so the final pool may be unchanged — but this needs to be measured on real models.Task
Compare, with vs without the
empty-tensor-to-alloc-tensorpass, on:Metrics:
hipdnn.pool_size— final GPU pool bytes stamped by--hip-pool-allocs(most direct).memref.alloccount.Method
Dump per-pass IR (
HIPDNN_EP_IR_DUMP_PATH+HIPDNN_EP_IR_DUMP_TREE=1) for each model on a build with the pass and a build without it; diff the two metrics.Outcome
Quantify the delta. If the pool/buffer growth is material, consider a more surgical alternative (e.g. only convert empties that feed a DPS
outs, or re-materialize distinct destinations after CSE) instead of converting all empties.Refs: PR #384, comment #384 (comment)