The massive amount of human biological, imaging, and clinical data produced by multiple and diverse sources necessitates integrative modeling approaches able to summarize all this information into answers to specific clinical questions. In this paper, we present a hypermodeling scheme able to combine models of diverse cancer aspects regardless of their underlying method or scale. Describing tissue-scale cancer cell proliferation, biomechanical tumor growth, nutrient transport, genomic-scale aberrant cancer cell metabolism, and cell-signaling pathways that regulate the cellular response to therapy, the hypermodel integrates mutation, miRNA expression, imaging, and clinical data. The constituting hypomodels, as well as their orchestration and links, are described. Two specific cancer types, Wilms tumor (nephroblastoma) and non-small cell lung cancer, are addressed as proof-of-concept study cases. Personalized simulations of the actual anatomy of a patient have been conducted. The hypermodel has also been applied to predict tumor control after radiotherapy and the relationship between tumor proliferative activity and response to neoadjuvant chemotherapy. Our innovative hypermodel holds promise as a digital twin-based clinical decision support system and as the core of future in silico trial platforms, although additional retrospective adaptation and validation are necessary.

The annotation of the tumour from medical scans is a crucial step in nephroblastoma treatment. Therefore, an accurate and reliable segmentation method is needed to facilitate the evaluation and the treatments of the tumour. The proposed method serves this purpose by performing the segmentation of nephroblastoma in MRI scans. The segmentation is performed by adapting and a 2D free hand drawing tool to select a region of interest in the scan slices. Results from 24 patients show a mean root-mean-square error of 0.0481 ± 0.0309, an average Dice coefficient of 0.9060 ± 0.0549 and an average accuracy of 99.59% ± 0.0039. Thus the proposed method demonstrated an effective agreement with manual annotations.

In many biomedical applications, it is often desired to simulate, analyse and visualise the dynamics of a particular patient based on a patient-specific musculoskeletal model. However, reconstructing a patient-specific model directly from medical images is highly labour intensive, and impractical in the clinical context. A more efficient method is to derive it from an atlas musculoskeletal model using patient-specific hints. In this paper, Laplacian mesh processing is introduced to deform an atlas model to a patient-specific model, based on patient-specific landmarks extracted from two orthogonal clinical images and using least-squares error optimization. Muscle attachment landmarks and motion landmarks in the atlas are also transformed as part of the process. Drift and inter-surface penetrations are prevented by supplementary inter-surface landmarks. Mesh simplification and reconstruction are used to avoid out-of-memory failures that may result from trying to deform models at high resolution.

Vascular stent deployment in peripheral arteries is a medical intervention in which a wire mesh tube is inserted into the artery to provide internal support. However, stents positioned in locations such as the femoral artery are subject to cyclic bending, and are therefore at risk of fatigue fracture. A software tool chain, called AimaSimul, is being implemented to support stent modeling, surgical simulation and risk calculation for surgical planning. In particular, the AimaSimul preoperative planning tool allows clinicians, starting from patient-specific medical images, to interactively assess different stent models and deployment options for the risk of breakage. This paper describes the main functionalities of AimaSimul and, in particular, the stent deployment and deformation.

This paper proposes a way to develop high-quality real-time web-based medical 3D data visualisation. Isosurface extraction is used as an example to discuss how to use programmable Graphics Processing Units (GPUs) and shaders to improve rendering performance on the web. The method is designed to reduce data transmission. When data is ready, the performance penalty can be considered negligible. A method to estimate memory usage to balance client memory limitation and rendering quality is also described. A way of using the frame rate to measure performance on the web is suggested, which could be used in future web visualisation.

The MSV project aims to survey current best practice in multiscale visualisation and to construct a software toolkit which will make multiscale techniques readily accessible to biomedical researchers and clinicians. In this paper, current methods for multiscale data visualisation in several domains are reviewed, and a novel classification of multiscale techniques for biomedical applications by function is proposed. The classification will form the basis of a design menu and toolkit for multiscale visualisation.

This paper proposes an easy way to perform web-based 3D interactive visualisation, which accepts two data resources, local and remote, as input and copes with two types of algorithm, built-in and remote. The focus of this paper is the biomedical area. Today's web technology makes it possible to use software maintained on a remote server. The approach described provides a virtual client environment, in which users can employ remotely installed software interactively using any standard browser. The proposed approach is a generic one and can be used in many other application areas.

Ischaemic heart failure remains a significant health and economic problem worldwide. This paper presents a user-friendly software system that will form a part of the virtual pathological heart of the Virtual Physiological Human (VPH2) project, currently being developed under the European Commission Virtual Physiological Human (VPH) programme. VPH2 is an integrated medicine project, which will create a suite of modelling, simulation and visualization tools for patient-specific prediction and planning in cases of post-ischaemic left ventricular dysfunction. The work presented here describes a three-dimensional interactive visualization for simulating left ventricle restoration surgery, comprising the operations of cutting, stitching and patching, and for simulating the elastic deformation of the ventricle to its post-operative shape. This will supply the quantitative measurements required for the post-operative prediction tools being developed in parallel in the same project.

publication. Citations to Advance online articles must include the digital object identifier (DOIs) and date of initial online articles are citable and establish publication priority; they are indexed by PubMed from initial publication. the paper journal (edited, typeset versions may be posted when available prior to final publication). Advance Advance online articles have been peer reviewed and accepted for publication but have not yet appeared in

While there has been significant progress in the treatment of ischemic heart failure, it remains a significant health and economic problem worldwide. In this paper, we present the challenges of modelling ischemic heart failure and introduce a user-friendly software system that will be a sub-set of the Virtual Pathological Heart environment which is currently being developed under the FP7 VPH2 project. This will provide patient-specific computational modelling and simulation of the human heart to assist the cardiologist and the cardiac surgeon in defining the severity and extent of disease in patients with post-ischemic Left Ventricular Dysfunction. The proposed system will provide visualisation tools for surgical assessment and planning: the registration and display of necrotic and hypo-kinetic regions; simulated surgical restoration (cutting and patching); and finally post-operative functional prediction (volume, shape and mitral valve regurgitation).

The introduction of integrative approaches to biomedical research (integrative biology, physiome, Virtual Physiological Human, etc.) poses original problems to computer aided medicine: the need to operate with large amounts of data that are strongly heterogeneous in structure, format and even in the knowledge domain that generated them; the need to integrate all of these data into a coherent whole; the further complication imposed by the fact that more and more frequently these data are captured at very different dimensional and/or temporal scales. The present study describes a first attempt at providing an interactive visualisation environment for homogeneous biomedical data defined over radically different spatial or temporal scales. In particular, we describe new strategies for the management of the dimensional information of highly heterogeneous data types; the management of temporal multiscaling; for 3D unstructured spatial multiscale visualisation and the related interaction paradigms and user interface. Preliminary results with a prototype implementation based on the OpenMAF application framework (http://www.openmaf.org) indicate that it is possible to develop effective environments for interactive visualisation of multiscale biomedical data.

In this paper, we present an approach to interactive out-of-core volume data exploration that has been developed to augment the existing capabilities of the LhpBuilder software, a core component of the European project LHDL ( http://www.biomedtown.org/biomed_town/lhdl ). The requirements relate to importing, accessing, visualizing and extracting a part of a very large volume dataset by interactive visual exploration. Such datasets contain billions of voxels and, therefore, several gigabytes are required just to store them, which quickly surpass the virtual address limit of current 32-bit PC platforms. We have implemented a hierarchical, bricked, partition-based, out-of-core strategy to balance the usage of main and external memories. A new indexing scheme is introduced, which permits the use of a multiresolution bricked volume layout with minimum overhead and also supports fast data compression. Using the hierarchy constructed in a pre-processing step, we generate a coarse approximation that provides a preview using direct volume visualization for large-scale datasets. A user can interactively explore the dataset by specifying a region of interest (ROI), which further generates a much more accurate data representation inside the ROI. If even more precise accuracy is needed inside the ROI, nested ROIs are used. The software has been constructed using the Multimod Application Framework, a VTK-based system; however, the approach can be adopted for the other systems in a straightforward way. Experimental results show that the user can interactively explore large volume datasets such as the Visible Human Male/Female (with file sizes of 3.15/12.03 GB, respectively) on a commodity graphics platform, with ease.

The Visualisation ToolKit (VTK) has become a very popular tool for scientific data visualisation and it is used as a base in many existing visualisation systems. Scientific datasets produced nowadays by complex scientific simulations or by modern data acquisition techniques (e.g., airborne laser scanning) are often too large to be processed in one piece on commodity hardware, as simply storing it requires several giga-bytes. Although VTK provides a means for processing such datasets, their straightforward use is rarely efficient. This paper describes an efficient interactive exploration of large volume datasets under the Multimod Application Framework (MAF) [VZT ∗ 07], a VTK-based system; however, the proposed approach can be adopted for other systems with ease. It exploits various techniques such as multi-resolution layout and a volume bricking scheme to access data at an interactive rate. A user can explore the dataset by specifying a region of interest (ROI), which leads to the generation of a more accurate data representation inside the ROI. If even more precise accuracy is needed inside the ROI, nested ROIs are used. Experimental results show that the user can interactively explore large volume datasets such as the Visible Human male (1760x1024x1878, with a file size of 3.15 GB) on a commodity platform. Graphics

A 3D multiscale view has been developed for medical datasets that are composed of multiple images of very different spatial sizes. The example data consisted of 3 volume images of bone: a 400 mm femur; a 20 mm micro-CT image of trabeculae; and a 0.5 mm nano-CT image of a single trabecula. The images were nested, the nano-CT being inside the micro-CT, which was within the head of the femur. The images were visualised using a volume slice view. The dataset was multiscale in that each image could not be usefully resolved at the scale of the next larger image. The multiscale interaction within the system is based on the replacement of sub-scale images by small click-and-zoom tokens. Issues relating to interaction with the tokens are discussed and early outcomes from this form representation are described.

A high-resolution 3D stereo photogrammetry system has been developed and it succeeded in capturing 3-D shapes of live pigs for pig husbandry research. For investigation of relationship between diet and growth of the pigs, 32 pigs were divided into high-lysine and low-lysine diet groups. Each pig was imaged once per week for 14 weeks. The system consists of three stereo pairs, which capture stereo images from side, top and rear views. Image processing software has been developed which included camera calibration, stereo image matching and range image integration to reconstruct 3D shapes of the pigs for animal science analysis. A conformation algorithm was proposed to deform a generic leg model to the shapes of pigs to extract quantitative ham volume weekly measurements from the shapes. The system has been proven to be accurate and reliable during the experiments.

A photogrammetric stereo imaging system was used to capture 3-D models of live pigs, and quantitative shape measurements were extracted from cross sections of the models. Stereo images were captured of 32 pigs, divided into high-lysine and low-lysine diet groups, and 3-D models were built from the images. Each pig was imaged once per week for 14 weeks. After slaughter, 10 of the pigs were dissected for muscle and fat measurements. A sequence of algorithms was applied to the 3-D models : differential geometry to reveal surface curvature features and detect the spine; manual landmark placement; fitting a curve to the spine; determining the vertical axis of the body; placing a slice plane across the abdomen close to the P2 position; extracting a cross section; and fitting a shape model to the cross section. Differential geometry revealed many qualitative features of the musculature. The spine was a line of minimum curvature along the back. The high-lysine pigs had higher height-to-width ratios and flatter backs than the low-lysine pigs. The dissected total muscle mass had a -0.66 correlation with the flatness-of-back shape parameter, and a 0.64 correlation with weight.

The value of an animal for breeding, production or slaughter depends on its body composition (the distribution and relative amounts of fat and muscle). Technology to measure or predict these variables would enable the producer to control them by regulating the animal's diet and to improve the selection of animals for breeding, production and for slaughter. This paper examines the progress that is being made in the development of technology for measuring or predicting live animal composition on the farm. This technology includes image analysis, palpation and robotics for sensor placement.