In advanced setting of root configuration (to be inherited by all configurations) define a node assignment rule specifying a rule
node.getAttribute("nodeError") != null
And in then edit post-build script of root configuration (again to be inherited by all configurations), define a script like this:
groovy:
for (step in build.steps) {
if (step.errorMessage != null && step.errorMessage.contains("OutOfMemoryError"))
step.node.setAttribute("nodeError", step.errorMessage);
}
This way the node in error will be excluded from the grid, and it can only be used after you fix the issue and delete the nodeError attribute manually from the affected node.