Once we decide to project features from a 3D point cloud onto 2D planes, one design seems natural: use all three—XY, XZ, and YZ. More views should preserve more information.
The ablation experiments in PlaneSegNet produced a less intuitive result. Adding only the horizontal XY projection did not improve segmentation at the plant–non-plant boundary; performance declined slightly. Either vertical projection, XZ or YZ, helped. Combining all three planes was again worse than keeping only the two vertical ones.
I find this phenomenon more memorable than the model’s final score. It reveals a lesson that extends beyond a particular module: a representation does not passively transport information; it actively selects which relationships deserve to survive.
Voxelization Only Answers “Where Do We Compute?”
A raw point cloud is a set of coordinates scattered through 3D space, without the regular pixel grid of an image. Voxelization assigns those points addresses in a regular 3D lattice. Sparse convolution then computes only at occupied voxels instead of paying mostly for empty space. PlaneSegNet is built around a sparse 3D U-Net and retains the voxel-to-point index mapping so that voxel predictions can eventually be returned to the original points.
But being able to compute efficiently does not mean we have found the relationships worth computing.
The boundary between a plant and the ground is particularly difficult. Stems and leaves may lie close to the soil, while weeds and mechanical structures can appear nearby. Shape inside a local group of voxels does not always provide a clean answer. Sparse convolution can expand its receptive field layer by layer, but ambiguous locations still need a way to receive larger-scale geometric context.
Plane projection is not merely a way to turn a 3D object into three easier images. It asks a more consequential question: when gathering distant information, which coordinate are we willing to discard temporarily?
Every Projection Is Directional Compression
Aggregating 3D voxel features at matching 2D coordinates must collapse one direction. Projection onto XY collapses height z; projection onto XZ collapses depth y; projection onto YZ collapses width x.
The three planes are not equivalent views of the same object, because each throws away different information.
For separating plants from ground, height is unusually informative. From above, leaves, stems, and ground clutter can occupy similar horizontal positions, so points at different heights are overlaid. In a vertical plane, the ground becomes a narrow low band while the plant’s vertical extension remains visible. XZ and YZ preserve that height variation from two orthogonal directions, reducing the blind spots of a single side view.
Elevated plant points collapse onto the cluttered ground footprint.
The plant remains elevated above a thin ground band.
A second vertical view preserves the same height transition.
The figure does not simulate learned attention weights. It retains only the elementary geometry of projection. The set of points never changes; the difference is what each flattening operation discards. For this task, preserving height matters more than collecting every view.
More Inputs Do Not Guarantee More Usable Information
In the literal sense, concatenating XY features gives the network more numbers. But more numbers and more discriminative evidence are not the same thing.
PlaneSegNet’s plane block mean-aggregates 3D features at each projected location, aligns the resulting 2D features back to voxel space, and fuses them with the original 3D features. This lets distant voxels sharing plane coordinates interact, but it also compresses differences along the projected axis. The horizontal projection collapses height precisely where ground clutter and plant points often overlap in horizontal position, producing a more entangled summary.
The following concatenation and multilayer perceptron do not have unlimited capacity to ignore every irrelevant input automatically. An unhelpful view can compete with the useful vertical cues and make it harder for the fused representation to organize itself around the task. The ablation result—three planes underperforming XZ+YZ—provides a testable piece of support for that claim.
This does not prove that XY projection is inherently harmful. For road boundaries, roof outlines, or a different coordinate convention, horizontal structure may be the decisive signal. The experiment supports a narrower and more reliable conclusion: attention does not decide what deserves attention for us; it amplifies relationships that the representation has already made visible.
If the Difference Lives at a Boundary, Do Not Measure Only the Whole
The other half of this intuition comes from evaluation.
Plant interiors and broad areas of ground are usually easy to classify. Plane orientation mostly affects the narrow region where the two classes meet. Yet boundary points make up only a small part of an entire point cloud. A global intersection-over-union or F1 score lets numerous easy points overwhelm a local improvement, making several models appear nearly identical.
The paper therefore starts at the plant’s lowest height and divides the space along the Z axis into six 0.05-meter layers, evaluating the region close to the ground separately. The difference among XZ, YZ, and XY becomes visible only when the measurement is aimed at the location the method claims to improve.
This is the same question as choosing a projection plane. Representation determines what the model preserves; evaluation determines what the experiment can see. If a method claims to improve boundaries but reports only a global metric dominated by interior points, a higher number still does not test its central claim.
Research Should Leave Behind an Intuition That Survives Questions
As one of the paper’s authors, I do not think the most valuable material for a blog is the parameter count, inference time, or list of datasets on which one model surpassed another. Those numbers are necessary evidence for the paper, but they change quickly with hardware, implementation details, and industrial optimization.
What lasts longer is an intuition constrained jointly by a method and its experiments: organize sparse 3D computation in voxels, then project the geometric relationship on which the task depends. Do not assume all directions are equivalent, and do not assume that giving every feature to a network means the network will make the choice for us. Finally, evaluation has to magnify the region that this choice actually changes, or the experiment validates only an average.
The value of research does not have to be a final answer ready for deployment. It can turn a vague experience into an explicit hypothesis, then design a structure and an experiment that give the hypothesis a chance to fail. To me, the most valuable part of PlaneSegNet is not the statement that “plane attention works,” but a more precise idea: when a task depends on directional structure, ignoring one irrelevant direction can bring us closer to understanding than collecting one more view.
Paper and code: Xin Yang, Chenyi Xu, et al., “PlaneSegNet: A deep learning network with plane attention for plant point cloud segmentation in agricultural environments”, Artificial Intelligence in Agriculture; project source code.