A tool is only as good as its user

This is a very interesting paper from Toronto. The aim is clear, and the methodology is sound. They use machine learning to increase the predictive capacity of ultrasound markers in children with PUJ obstruction (PUJO) and improve the criteria to determine in which children diuretic renogram can be avoided [1]. Machine learning is a fascinating tool; it builds on the principle of algorithms and decision-trees with the ability to self-improve. In our medical field, it reduces the ‘noise’, or unwanted variability, in clinical decision-making. It can provide a reliable, reproducible easily usable aide. Nevertheless, there are limits too. The main ones being that machine learning and artificial intelligence (AI) are only as good as the data they receive, and that AI does not think. This is why this study is so interesting. It shows both the advantages and some of the limits of AI. The authors’ set out to develop a model with high sensitivity for obstruction and improve the decisions that would be made if based on Society for Fetal Urology (SFU) grade or anteroposterior diameter alone. Need for pyeloplasty was a secondary outcome, with indications including abnormal differential function at baseline or prolonged drainage. In reality, the authors’ developed a model with high-sensitivity for impaired drainage on MAG3 scan not a model for obstruction or need for surgery. Neither delayed drainage nor undergoing a pyeloplasty are evidence of obstruction. The authors’ show an overlap between drainage time and surgery, but this is hardly surprising as they consider the first an indication for the second. As the authors’ state, prolonged drainage on MAG3 can indeed place patients in risk categories, but it does not identify obstruction, nor does it, for many teams, constitute sufficient argument to proceed to pyeloplasty. To illustrate this, I suggest reading again the article by Onen et al. [2] on conservative management of children with unilateral hydronephrosis, where all patients were initially treated non-operatively regardless of the initial diuretic renogram washout pattern. Decision to intervene depended on the child's evolution rather than his initial findings, and this evidence was obtained through serial measurements. An obstructed washout pattern (>20 min) was present in 58% of children who were successfully followed non-operatively. A year later, Amarante et al. [3] published the very interesting paper titled: ‘Impaired drainage on diuretic renography using half-time or pelvic excretion efficiency is not a sign of obstruction in children with a prenatal diagnosis of unilateral renal pelvic dilatation’. The point being that too many children are considered obstructed and undergoing surgery, when they are just in a risk group that warrants surveillance. To operate or to observe? As shown by the Cochrane systematic review as well as the recommendations of the European Association of Urology-European Society for Paediatric Urology (EAU-ESPU) guidelines, the question remains open [4, 5]. In this regard, the authors should be commended for proposing a model that decreases the use of invasive investigations. Their model proposes to perform fewer diuretic scans. This is good. However, the next question is why were they being prescribed in the first place? The authors follow their national recommendations, which are based on SFU grade. These recommendations can be questioned. Investigations must be disease specific. Hydronephrosis is a term, which lacks precision, and description of the pelvis and ureter should direct need for further investigations. There has fortunately been a decrease in the use of systematic voiding cystourethrogram, when the ureter is normal, but we can probably also decrease the use of systematic nuclear scans. The old fear that major loss of function would happen under our eyes if we did not investigate early enough is overrated and it is probable that simply repeating ultrasound scans would identify those patients who will most benefit from renal scans. Long-term assessment of prenatally detected anomalies has shown that for fear of allowing any degree of loss of renal function, many children are over investigated and over treated. The study published by Arena et al. [6] in 2018 showed that 67% of urinary tract dilatation (UTD) Grade P2 and 50% of UTD Grade P3 did not require surgery and were treated conservatively, without loss of differential renal function or onset of symptoms related to PUJO. The machine-learning model described in this study [1] does improve the relevance of asking for a nuclear scan. However, it is possible that it does so in a population where nuclear scans are overprescribed, because based on another simpler algorithm. Models decrease the variability and disparities in management that can result from practitioner to practitioner, but they can also annihilate the specifics of each patient. Like every tool, they are only as good as their users and should be used with caution. Simply repeating the ultrasounds could also be a viable option. There is no funding. None declared.