Once a maize field becomes a point cloud, the hardest points to identify often lie close to the ground. High leaves stand out, and broad patches of bare soil are easy to recognize. The trouble comes from drooping leaf tips, stem bases, and scattered ground points between them. Within a small neighborhood, their shapes and heights can look remarkably similar. A person rotates the view to see what larger structures they belong to. To understand PlaneSegNet, start with that gesture.
The paper adds two views to a 3D segmentation network: XZ and YZ, two side planes that preserve height. In the ablation study, adding the overhead XY plane makes boundary segmentation worse. Three directions seem more complete than two. Why does that extra view fail to help?
Height separates the two classes
Place the Z axis vertically and the difference becomes intuitive. Projecting onto XY presses the leaves above and the soil below into the same overhead view, removing their separation in height. In XZ or YZ, the ground roughly forms a low band, with plants extending upward from it. A leaf close to the soil remains difficult to classify, but the model gains another clue: does this point fit the surrounding ground, or the downward continuation of a plant? The three views below show the same simplified scene, letting you compare what happens when each coordinate axis is discarded.
Elevated plant points collapse onto the cluttered ground footprint.
The plant remains elevated above a thin ground band.
A second vertical view preserves the same height transition.
Height matters because plants inhabit a world with a special direction: gravity. Ground spreads horizontally while plants generally grow upward, giving the vertical direction structural information that differs from the horizontal directions. Both side planes retain height, while each retains a different horizontal direction, making the views complementary. The three coordinate axes have equal status in a formula, but different roles in this maize field.
Send the projected information back into 3D
PlaneSegNet retains a sparse 3D U-Net backbone. Points first enter a grid of small cubes, or voxels. The network extracts features at occupied cells, progressively widens its view, and then restores resolution. Plane modules sit within this encoding and decoding process, so they operate on 3D features at multiple scales. Saved indices ultimately map voxel classifications back to the original points. The network diagram in the project repository shows this backbone alongside the plane module inside it.
Follow one voxel to see what a plane module does. For the XZ projection, 3D features that land at the same XZ grid location are averaged along Y to produce a plane feature. The module then uses the original coordinates to assign it back to the corresponding 3D voxels. The YZ branch repeats the operation in the other direction. Each voxel now has its local features together with information gathered from two directions. Concatenation, a two-layer fully connected network, and a residual connection produce the enhanced representation.
Voxels far apart along one direction now have a direct route for sharing information. The 3D branch keeps local detail, while the plane branches provide directional summaries; the dimension lost in a projection has not been deleted from the whole network. This organization makes structure at similar heights easier to access, without relying entirely on local convolutions to pass distant clues through successive layers.
Measure where the task is hardest
One easily overlooked detail in the plane ablation study is how close the variants are in overall IoU and F1. Many elevated plant points and unambiguous ground points are already classified correctly, leaving difficult regions as a small share of the scene. The authors therefore add a height-based evaluation: starting at the lowest plant point, they examine six layers in 5-centimeter increments to isolate performance near the ground. A method designed to improve boundaries needs to be assessed where those boundaries are.
Within this narrower region, the differences between directions become visible. The table summarizes the six configurations in the paper’s ablation study. The results fit the geometric intuition: the two height-preserving views help, while the extra overhead aggregation harms boundary performance. Having information in the input does not make every way of summarizing it useful to the model.
| Plane configuration | Boundary performance |
|---|---|
| No plane module | Sparse 3D U-Net baseline |
| XY only | Slightly worse than the baseline |
| XZ only; YZ only | Both single-plane variants improve performance |
| XZ + YZ | Best of the six, with especially clear gains near the bottom |
| XY + XZ + YZ | Worse than using the two vertical planes alone |
When the ground survives only in fragments
As crops mature, a dense canopy hides the ground, leaving sparse, disconnected ground returns in the scan. The cue of finding a flat patch nearby becomes weaker too. The paper tests these scenes directly with weights trained at an earlier growth stage, without further fine-tuning: ground IoU is 0.6613 for the sparse 3D U-Net and 0.9774 for PlaneSegNet. IoU measures the intersection of predicted and labeled ground against their union; these results show that the model recovers more ground fragments surrounded by plants in the tested scenes. The sparse-ground experiment helps explain how, when local evidence is scarce, distant structure at similar heights can still provide a reference.
This explanation also reveals a practical condition: the coordinate system should represent the vertical direction. The authors tilt the point clouds by 5, 10, and 15 degrees around the X and Y axes to mimic sensor mounting or orientation errors. Boundary accuracy declines, but remains above the sparse 3D U-Net baseline. Modest tilt does not immediately erase the useful structure, while the decline shows that direction is doing real work. Aligning “up” in the point cloud is therefore a natural deployment step.
The paper also trains and evaluates the network on Crop3D greenhouse data containing maize, rapeseed, and potato, and on the DALES airborne point-cloud dataset. DALES requires distinguishing eight classes, extending the task beyond plant-versus-background segmentation. These experiments take the design beyond a single maize field: ground, vegetation, and buildings still offer useful vertical structure. Direct tests on external maize datasets address a different question: whether already learned features transfer across data sources.
What I like about this design is how it turns a small piece of human experience with point clouds into an explicit route for information. From the side, the relationship between ground and plants becomes easier to see. Retain summaries from those two directions, let the 3D features supply the detail, and the network gains a useful way to observe the scene. All the coordinates were already present in the point cloud. Choosing a projection makes the relationships needed for the task easier to find.