Skip to main content

Nonlinear observer output-feedback MPC treatment scheduling for HIV

Abstract

Background

Mathematical models of the immune response to the Human Immunodeficiency Virus demonstrate the potential for dynamic schedules of Highly Active Anti-Retroviral Therapy to enhance Cytotoxic Lymphocyte-mediated control of HIV infection.

Methods

In previous work we have developed a model predictive control (MPC) based method for determining optimal treatment interruption schedules for this purpose. In this paper, we introduce a nonlinear observer for the HIV-immune response system and an integrated output-feedback MPC approach for implementing the treatment interruption scheduling algorithm using the easily available viral load measurements. We use Monte-Carlo approaches to test robustness of the algorithm.

Results

The nonlinear observer shows robust state tracking while preserving state positivity both for continuous and discrete measurements. The integrated output-feedback MPC algorithm stabilizes the desired steady-state. Monte-Carlo testing shows significant robustness to modeling error, with 90% success rates in stabilizing the desired steady-state with 15% variance from nominal on all model parameters.

Conclusions

The possibility of enhancing immune responsiveness to HIV through dynamic scheduling of treatment is exciting. Output-feedback Model Predictive Control is uniquely well-suited to solutions of these types of problems. The unique constraints of state positivity and very slow sampling are addressable by using a special-purpose nonlinear state estimator, as described in this paper. This shows the possibility of using output-feedback MPC-based algorithms for this purpose.

Background

The majority of untreated HIV patients, following a brief period of acute infection, enter a long asymptomatic phase of infection characterized by high viral loads, persistent immune activation, and a slow decline in the helper-T cell concentration [1]. Eventually, the concentration of helper-T cells becomes too low to sustain effective immune responses, and opportunistic infections cause a dramatic decline in the patient's health. The slow decline of helper-T cells during the asymptomatic phase was once thought to indicate a slow rate of infection and cell turnover, but it is now known that very fast rates of virus and host cell turnover, as high as 1010 virions per day or 2 × 109 infected helper-T cells per day occur during this phase [2, 3].

The majority of patients follow the disease progression pattern described above, but a small number of untreated patients, termed Long-Term Non-Progressors, do not show progressive decline in helper-T cell counts, have consistently low measured viral loads, and do not show impaired immune responses, and show strong HIV-specific helper-T cell responses [4, 5]. Levels of Cytotoxic-T cells specific to HIV in these patients remain high, even at low viral loads [6, 7]. Patients with progressive HIV infection show a dramatic drop in the level of these cells when the viral load is reduced [8, 9]. Long-Term non-progressors can transition to progressive HIV infection [10], probably due to the evolution of HIV strains resistant to the immune response [11].

In order to prevent mutation escape of the virus, HIV therapy uses three antiviral drugs simultaneously. These drugs, which target different epitopes in the HIV genome, make it very unlikely that the virus can simultaneously evolve resistance to all three drugs. This approach, called Highly Active Anti-Retroviral Therapy (HAART) is very effective at reducing viral load [12]. Unfortunately, the drugs used in HAART have a number of significant adverse side effects, and must be continued for the life of the patient [13]. HAART interruptions have been investigated in order to manage side effects of treatment or to allow treatment of secondary infections such as hepatitis-A [14–16]. A small number of cases where therapy was started during acute infection and then discontinued and re-initiated have apparently led to long-term, drug-free suppression of the virus [17, 18]. Follow-up studies investigating structured treatment interruptions (STI) as a method of inducing immune-mediated control of the virus showed some success in inducing a transient immune-mediated control of the virus [19–23]. Patients showing viral control also showed increased HIV-specific helper-T cell counts and increased HIV-specific cytotoxic-T cell counts, similar to the pattern seen in LTNPs. Follow-up studies tracking these patients showed that a majority of these patients eventually reverted to an actively progressing infection [24].

Studies of STI in patients who originally initiated treatment during chronic infection showed no success in inducing immune-mediated control, suggesting that treatment initiation during acute infection is a necessary condition for success in this approach [25–33]. HIV is known to preferentially infect HIV-specific T-cells [34], so HIV-specific helper-T cell pools may be permanently damaged in patients that delay therapy until the chronic phase of infection [35–39].

The use of STIs in HIV therapy is controversial [40]. Interruptions in therapy are likely to encourage the evolution of drug-resistance mutations [41–43]. It is clear that before these STI-based methods will be attempted again, a reliable model of resistance risk will need to be developed. This is the focus of much of our recent research [44–47]. Although STI-induced immune control has shown disappointing durability on its own, it could still be used in conjunction with a reduced-dosage HAART to attain similar levels of viral suppression with fewer side effects. Assuming that the immune response affects different targets from the HAART, this regimen should also be more durable than HAART alone. Some evidence exists for the possibility of durable immune control, as reported in [48]. Nevertheless, it will be necessary to increase the success rate of STI in inducing immune-mediated control, and find methods of moderating the risk of resistance evolution, for this method to become a viable option for HIV therapy.

In previous work, we developed a Model Predictive Control (MPC) based method for finding these schedules. This method is well-suited to the problem for a number of reasons: It is easily adaptable, which will allow for various improved models to be integrated as they are developed. It inherits from the MPC framework a certain robustness to disturbances and model inaccuracies which is important, since the model in question is known to suffer from these. It allows us to fine-tune the treatment using medically intuitive notions of cost. Finally, the long time-scales of the model allow us to overcome the computation time issues which normally plague MPC-based methods. However, the original work in [49] assumed full-state measurements. In practice only viral load measurements can be made with the frequency and accuracy necessary for a feedback control method.

In this paper, we introduce a full nonlinear observer with acceptable properties, and test its reliability in the face of model uncertainty. This serves as a "proof of concept" study for the use of nonlinear-observer output-feedback MPC in treatment scheduling for HIV. Other authors have also considered similar problems. The authors of [50] introduced an output-feedback model predictive control-based method for treatment scheduling for a different but related model of HIV dynamics, using an Extended Kalman Filter as the observer. The performance of this estimator began to rapidly degrade with model parameter uncertainty; however, a one-to-one comparison is not possible as the model of HIV dynamics was not the same. The authors of [51, 52] also developed an output feedback MPC algorithm for treatment scheduling of a different model of HIV infection; this paper used a deadbeat observer and assumed the ability to measure both CD4+ T-cell count and viral load, instead of just viral load as in this paper. The authors of [53] considered open-loop finite-horizon optimal control of a very simple model of HIV infection, allowed continuous varying of drug concentration, and did not consider either the measurement problem or the model inaccuracy problem. The authors of [54] introduced a robust multirate MPC controller to calculate treatment schedules for a model of HIV infection that does not include immune response dynamics, allowing continuous variation of drug dosing. The authors of [55, 56] developed an innovative output feedback scheduling method for the same model which we use, but assume that both the CD4+ T-cells and viral load are measurable. Their method does not use an MPC scheduling method. The authors of [57] present an output-feedback method for controlling a variation of the model which we use; however, their approach allows for continuous values of drug dosing, unlike our method which assumes constant drug dosing of either 1 or 0. The authors of [58] introduced a sophisticated nonlinear observer design for the same HIV model used in this work, with very good convergence in the continuous measurement case. Their method, however, relied on direct estimates of higher-order derivatives, which required sampling every 6 hours during the early phase of treatment, compared with the 1-week intervals proposed in this work. The authors of [59] consider the treatment scheduling problem as a multi-objective optimization and obtain a Pareto frontier, incidentally showing the near-optimality of our previously reported results in [49]. The authors of this paper do not consider the problem of output feedback. A review of the various control approaches applied to HIV medicine was presented in [60].

The paper is organized as follows. We first introduce the Wodarz-Nowak model of HIV infection. Next we introduce the nonlinear observer design. We then show the performance of this observer with continuous measurements and sampled measurements. Next we introduce the complete output-feedback MPC-based treatment scheduling method, which combines the full-state feedback MPC of [49, 61] with the observer introduced in the following section. Finally, we evaluate the performance of this design through Monte-Carlo experiments with various levels of model uncertainty. The conclusions discuss the implications of these results for future work in this area. This paper is the first to present a nonlinear-observer based output-feedback MPC algorithm for HIV treatment scheduling that incorporates realistic constraints on measurement intervals and relies only on the highly accurate viral load measurements.

Results and Discussion

Model

We consider a five-state nonlinear ODE model of HIV infection and immune response introduced in [62].

(1)

where x represents the concentration of healthy helper-T cells, y represents the concentration of HIV-infected helper-T cells, z 1 represents the concentration of inflammation mediated cytotoxic-T cells, w represents the concentration of memory phenotype cytotoxic-T cells, and z 2 represents the concentration of helper-T cell mediated cytotoxic-T cells. u is a binary variable representing the application of HAART, and η is HAART's effectiveness at reducing the infection rate. All states lie in the non-negative orthant, which is also positive invariant. u(t) is restricted to take values of either 0 (no treatment) or 1 (full treatment), in order to avoid the rapid evolution of the virus likely under suboptimally suppressed conditions. The measurable output, plasma viral load, is proportional to the infected cell state y, due to a singular perturbation phenomenon (the decay rate of the free virus is much faster than the death rate of infected cells). A more complete description of the states and their interactions can be found in our previous paper [49].

For realistic parameter values in the absence of treatment, this model has two stationary points in the non-negative orthant. One of these corresponds to a state where the virus is controlled by the immune response:

(2)

and one in which the virus dominates, progressing to AIDS:

(3)

This model in this paper uses normalized parameter values.

Observer Design

This application presents some unique challenges for observer design. The system described by Equation 1 is nonlinear with multiple steady-states. Observer design for such a system is very much an open problem. Also, the invasive nature of blood-drawing methods puts a very coarse lower limit on sampling time, with intersample times of one week a bare minimum. The observer must therefore be reasonably robust to error due to sampling.

After experimenting with simple high-gain type observers, we discovered they performed poorly in a sampled-data situation. The bare error injection term would, for large enough initial error, cause the observer to violate the non-negative orthant restriction of the original system, which wreaked havoc on the numerical simulator as well as being completely unrealistic. For the purpose of this paper, we settled on a nonlinear observer specific ally designed to satisfy the non-negative orthant restriction. The observer design was heavily influenced by the symmetry-preserving observer concept presented in [63], though we did not follow the same formal design approach. By starting with a copy of the system, and allowing output error-correction terms to enter the system in a manner following the system's natural geometry, we obtained an observer design that preserved state positivity, ensured good convergence, and avoided unrealistic patterns of intermediate state estimate values while the system was converging. The equations describing this observer are

(4)

where x e , y e , z 1e , w e , z 2e are the state estimates of x, y, z 1 , w, z 2 respectively. We let X refer to the vector of all states and X e refer to the vector of state estimates. This yields error dynamics described by the equations

(5)

where e X is X - X e . The combined system of Equations 1 and 5 have two steady-state solutions, corresponding to the steady-state values for the original system described in Equations 2 and 3 combined with the error values e X = 0. For the parameter values of Table 1, these two steady-states are locally stable by Lyapunov's second method.

Table 1 Parameter Values

Simulations

We implemented the observer described above, with model parameters as listed in Table 1, and tested its behavior under a variety of circumstances. With continuous feedback, the observer error converged asymptotically to zero for every combination of initial state and estimate. Representative examples of this can be seen in Figures 1 and 3. In Figure 1, the initial condition is in the same Region of Attraction (ROA) as the initial estimate (error convergence for this case is shown in Figure 2), and in Figure 3, the initial estimate is in a different ROA from the initial condition (error convergence for this case is shown in Figure 4). The location of initial conditions in a particular ROA was verified by simulation. In both cases the estimate converges to the actual value.

Figure 1
figure 1

Continuous-time observer convergence - same ROC. Initial estimate and initial condition are in the same Region of Attraction. Initial state is {x = 10, y = 0:1, z 1 = 0.1, w = 3, z 2 = 0.1}, initial estimate is {x e = 10, y e = 3, z 1e = 1, w e = 10, z 2e = 0:1}.

Figure 2
figure 2

Continuous-Time Observer Error Convergence. Initial Estimate and Initial Condition are in the same Region of Attraction.

Figure 3
figure 3

Continuous-time observer convergence - different ROC. Initial estimate and initial condition are in different Regions of Attraction. Initial state is {x = 10, y = 0.1, z 1 = 0.1, w = 3, z 2 = 0:1}, initial estimate is {x e = 2, y e = 3, z 1e = 1, w e = 0.1, z 2e = 0.1}.

Figure 4
figure 4

Continuous-Time Observer Error Convergence. Initial Estimate and Initial Condition are in different Regions of Attraction.

The actual system is constrained by a sampling period of no less than one week. Accordingly, we implemented in MATLAB a discretized sample and hold version of the continuous observer described above, with a sampling period of one week. Again, this observer performed well, with error converging toward zero, albeit at a slower rate than in the continuous-time case. Figure 5 shows the performance of the discretized observer when the initial condition is in the same ROA as the initial estimate (error convergence for this case is shown in Figure 6), and Figure 7 shows the performance when the initial condition is in a different ROA than the initial estimate (error convergence for this case is shown in Figure 8). It should be noted that the tuning parameters (K 1, K 2, K 3, K 4, K 5) used in these simulations are tuned for performance of the discretized observers. Better performance is possible for the continuous-time observer, at the cost of higher gains and worse performance for the discretized observer. The parameters (K 1, K 2, K 3, K 4, K 5) were tuned primarily through trial and error; local stability of the observer holds for a wide range of values, but there is a clear trade-off between convergence rate and sensitivity to low sampling rates.

Figure 5
figure 5

Discretized observer convergence - same ROC. Initial estimate and initial condition are in the same Region of Attraction. Initial state is {x = 10, y = 0:1, z 1 = 0.1, w = 3, z 2 = 0.1}, initial estimate is {x e = 10, y e = 3, z 1e = 1, w e = 10, z 2e = 0.1}.

Figure 6
figure 6

Discrete-Time Observer Error Convergence. Initial Estimate and Initial Condition are in the same Region of Attraction.

Figure 7
figure 7

Discretized observer convergence - different ROC. Initial estimate and initial condition are in different Regions of Attraction. Initial state is {x = 10, y = 0.1, z 1 = 0.1, w = 3, z 2 = 0.1}, initial estimate is {x e = 2, y e = 3, z 1e = 1, w e = 0.1, z 2e = 0.1}.

Figure 8
figure 8

Discrete-Time Observer Error Convergence. Initial Estimate and Initial Condition are in different Regions of Attraction.

Output Feedback

In this section, we present the output feedback adaptation of the Model Predictive Control based treatment scheduling algorithm of [49]. Output-feedback Model Predictive Control recursively solves a finite-horizon optimal control problem, applying the first sample of each optimal control solution before resampling the system. Consider a discrete-time system of the form

(6)

and a current measurable output y k = g(X k ) we find a length N sequence U = {u k , u k+1 , ..., u k+N-1 } which minimizes a cost function of the form

(7)

with stage cost l, terminal cost F, and state estimate X e obtained from the observer. The first element of the resulting optimal control sequence is applied, a new sample is obtained, and a new optimal control sequence is calculated. An excellent review of MPC in its various implementations can be found in [64]. Our control objective is to globally stabilize the stationary point described in Equation 2. We also want to minimize the transient decrease in helper-T cell concentration. We use the stage cost:

(8)

where α j > 0 are design weights and x LTNP , w LTNP are the desired equilibrium values for the respective states [65]. shows conditions on the full-state feedback system and controller which, if satisfied, guarantee robust asymptotic convergence to a neighborhood of the desired equilibrium. In a similar fashion, the work in [66] shows conditions on the system, output, observer, and state-feedback MPC formulation which, if satisfied, allow the use of the state-feedback MPC algorithm with the estimated state values from the observer to generate an output-feedback MPC algorithm which robustly stabilizes the desired steady-state. We implemented the output-feedback MPC algorithm described above in MATLAB. With no error in the model parameters, across a large range of randomly selected initial conditions and initial estimates, the algorithm always managed to stabilize a small neighborhood of the desired steady-state of Equation 2.

Robustness

While the measurements of viral load (proportional to output y) have well-characterized log-normal variation, the parameters of the system are expected to vary significantly from patient to patient, and are impossible to identify in practice for each patient. In [49], we characterized the robustness of the state-feedback system to error in the model. We introduced a at random variation into every parameter in the model. The scheduling algorithm continued to use the nominal, but now incorrect values to calculate its schedules. We ran at least 100 simulations each with this error randomly distributed at up to 5%, 10%, 15%, 20%, 25%, and 30% of each parameter value, allowing the algorithm up to two (patient) years to successfully stabilize the desired steady-state. The simulations were carried out from a common initial condition. We run the same Monte-Carlo type robustness test here on the output-feedback MPC algorithm described in this section, and the results are summarized in Table 2 with the results from [49] included for comparison. An example of typical performance is shown in Figure 9. As expected, the output-feedback algorithm was outperformed by the state-feedback algorithm, but the success rate of the output-feedback algorithm did not drop o dramatically as model error increased, and even at up to 30% error in every parameter, is still better than 70%. The inconsistencies between the 25% and 30% error cases are undoubtedly due to the small sample sizes, which in turn were forced by the computational cost of these simulations. These results demonstrate a practical robustness to modeling error which strongly motivates the use of output-feedback MPC in treatment scheduling for HIV.

Table 2 Robustness to modeling error
Figure 9
figure 9

Output Feedback MPC robust performance. Initial estimate and initial condition are the same, controller uses nominal parameters as in Table 1, parameters in the model vary randomly from these by as much as 10%.

Conclusions

In this paper, we have introduced a candidate nonlinear observer for use in output-feedback MPC-based treatment scheduling for HIV. The observer is designed to preserve the forward-invariance of the non-negative orthant in the face of sampling-induced measurement error. The observer performs well in both the continuous-time and discretized implementations.

We have implemented an output-feedback MPC-based scheduling algorithm, and tested its robustness to modeling error. The closed-loop system performed well. Also, the performance of this output-feedback system should be understood as a lower-bound on what is possible. This work motivates the use of output-feedback MPC, but the observer used is only one candidate observer. A more natural implementation might be a nonlinear receding-horizon observer as in [67], though the implementation of such an observer for a system such as ours is still an open problem.

The possibility of enhancing immune responsiveness to HIV through dynamic scheduling of treatment is exciting. Model Predictive Control is uniquely well-suited to solutions of these types of problems. The sample-and-prescribe framework is reconcilable to the realities of patient treatment through office visits. The work in this paper shows the possibility of using output-feedback MPC-based algorithms for this purpose.

References

  1. Koup R, Safrit J, Cao Y, Andrews C, McLeod G, Borkowsky W, Farthing C, Ho D: Temporal association of cellular immune responses with the initial control of viremia in primary human immunodeficiency virus type 1 syndrome. J Virol 1994, 68(7):4650–5.

    Google Scholar 

  2. Ho D, Neumann A, Perelson A, Chen W, Leonard J, Markowitz M: Rapid turnover of plasma virions and CD4 lymphocytes in HIV-1 infection. Nature 1995, 373(6510):123–6. 10.1038/373123a0

    Article  Google Scholar 

  3. Perelson A, Neumann A, Markowitz M, Leonard J, Ho D: HIV-1 dynamics in vivo: virion clearance rate, infected cell life-span, and viral generation time. Science 1996, 271(5255):1582–6. 10.1126/science.271.5255.1582

    Article  Google Scholar 

  4. Rosenberg E, Billingsley J, Caliendo A, Boswell S, Sax P, Kalams S, Walker B: Vigorous HIV-1-specific CD4+ T cell responses associated with control of viremia. Science 1997, 278(5342):1447–50. 10.1126/science.278.5342.1447

    Article  Google Scholar 

  5. Gloster S, Newton P, Cornforth D, Lifson J, Williams I, Shaw G, Borrow P: Association of strong virus-specific CD4 T cell responses with efficient natural control of primary HIV-1 infection. AIDS 2004, 18(5):749–55. 10.1097/00002030-200403260-00005

    Article  Google Scholar 

  6. Harrer T, Harrer E, Kalams S, Elbeik T, Staprans S, Feinberg M, Cao Y, Ho D, Yilma T, Caliendo A, Johnson R, Buchbinder S, Walker B: Strong cytotoxic T cell and weak neutralizing antibody responses in a subset of persons with stable nonprogressing HIV type 1 infection. AIDS Res Hum Retroviruses 1996, 12(7):585–92. 10.1089/aid.1996.12.585

    Article  Google Scholar 

  7. Migueles S, Laborico A, Shupert W, Sabbaghian M, Rabin R, Hallahan C, Baarle DV, Kostense S, Miedema F, McLaughlin M, Ehler L, Metcalf J, Liu S, Connors M: HIV-specific CD8+ T cell proliferation is coupled to perforin expression and is maintained in nonprogressors. Nat Immunol 2002, 3(11):1061–8. 10.1038/ni845

    Article  Google Scholar 

  8. McMichael A, Rowland-Jones S: Cellular immune responses to HIV. Nature 2001, 410(6831):980–7. 10.1038/35073658

    Article  Google Scholar 

  9. Mollet L, Li T, Samri A, Tournay C, Tubiana R, Calvez V, Debre P, Katlama C, Autran B: Dynamics of HIV-specific CD8+ T lymphocytes with changes in viral load. The RESTIM and COMET Study Groups. J Immunol 2000, 165(3):1692–704.

    Article  Google Scholar 

  10. Rodes B, Toro C, Paxinos E, Poveda E, Martinez-Padial M, Benito J, Jimenez V, Wrin T, Bassani S, Soriano V: Differences in disease progression in a cohort of long-term non-progressors after more than 16 years of HIV-1 infection. AIDS 2004, 18(8):1109–16. 10.1097/00002030-200405210-00004

    Article  Google Scholar 

  11. Yang O, Sarkis P, Ali A, Harlow J, Brander C, Kalams S, Walker B: Determinant of HIV-1 mutational escape from cytotoxic T lymphocytes. J Exp Med 2003, 197(10):1365–75. 10.1084/jem.20022138

    Article  Google Scholar 

  12. Gray C, Lawrence J, Ranheim E, Vierra M, Zupancic M, Winters M, Altman J, Montoya J, Zolopa A, Schapiro J, Haase A, Merigan T: Highly active antiretroviral therapy results in HIV type 1 suppression in lymph nodes, increased pools of naive T cells, decreased pools of activated T cells, and diminished frequencies of peripheral activated HIV type 1-specific CD8+ T cells. AIDS Res Hum Retroviruses 2000, 16(14):1357–69. 10.1089/08892220050140900

    Article  Google Scholar 

  13. Gegeny T: Surviving with HIV in the HAART era: emerging challenges. Res Initiat Treat Action 2000, 6(3):12–4.

    Google Scholar 

  14. Montaner L: Structured treatment interruptions to control HIV-1 and limit drug exposure. Trends Immunol 2001, 22(2):92–6. 10.1016/S1471-4906(00)01809-3

    Article  MathSciNet  Google Scholar 

  15. Dybul M, Chun T, Yoder C, Hidalgo B, Belson M, Hertogs K, Larder B, Dewar R, Fox C, Hallahan C, Justement J, Migueles S, Metcalf J, Davey R, Daucher M, Pandya P, Baseler M, Ward D, Fauci A: Short-cycle structured intermittent treatment of chronic HIV infection with highly active antiretroviral therapy: effects on virologic, immunologic, and toxicity parameters. Proc Natl Acad Sci USA 2001, 98(26):15161–6. 10.1073/pnas.261568398

    Article  Google Scholar 

  16. Fischer M, Hafner R, Schneider C, Trkola A, Joos B, Joller H, Hirschel B, Weber R, Gunthard HF, Study SHC: HIV RNA in plasma rebounds within days during structured treatment interruptions. AIDS 2003, 17(2):195–9. 10.1097/00002030-200301240-00009

    Article  Google Scholar 

  17. Autran B, Carcelain G: AIDS. Boosting immunity to HIV-can the virus help? Science 2000, 290(5493):946–9. 10.1126/science.290.5493.946

    Article  Google Scholar 

  18. Lisziewicz J, Rosenberg E, Lieberman J, Jessen H, Lopalco L, Siliciano R, Walker B, Lori F: Control of HIV despite the discontinuation of antiretroviral therapy. N Engl J Med 1999, 340(21):1683–4. [http://content.nejm.org/cgi/content/full/340/21/1683] 10.1056/NEJM199905273402114

    Article  Google Scholar 

  19. Lori F, Maserati R, Foli A, Seminari E, Timpone J, Lisziewicz J: Structured treatment interruptions to control HIV-1 infection. Lancet 2000, 355(9200):287–8. 10.1016/S0140-6736(99)03515-1

    Article  Google Scholar 

  20. Ortiz G, Nixon D, Trkola A, Binley J, Jin X, Bonhoeffer S, Kuebler P, Donahoe S, Demoitie M, Kakimoto W, Ketas T, Clas B, Heymann J, Zhang L, Cao Y, Hurley A, Moore J, Ho D, Markowitz M: HIV-1-specific immune responses in subjects who temporarily contain virus replication after discontinuation of highly active antiretroviral therapy. J Clin Invest 1999, 104(6):R13–8. 10.1172/JCI7371

    Article  Google Scholar 

  21. Papasavvas E, Ortiz G, Gross R, Sun J, Moore E, Heymann J, Moonis M, Sandberg J, Drohan L, Gallagher B, Shull J, Nixon D, Kostman J, Montaner L: Enhancement of human immunodeficiency virus type 1-specific CD4 and CD8 T cell responses in chronically infected persons after temporary treatment interruption. J Infect Dis 2000, 182(3):766–75. 10.1086/315748

    Article  Google Scholar 

  22. Ruiz L, Carcelain G, Martinez-Picado J, Frost S, Marfil S, Paredes R, Romeu J, Ferrer E, Morales-Lopetegi K, Autran B, Clotet B: HIV dynamics and T-cell immunity after three structured treatment interruptions in chronic HIV-1 infection. AIDS 2001, 15(9):F19–27. 10.1097/00002030-200106150-00001

    Article  Google Scholar 

  23. Rosenberg E, Altfeld M, Poon S, Phillips M, Wilkes B, Eldridge R, Robbins G, D'Aquila R, Goulder P, Walker B: Immune control of HIV-1 after early treatment of acute infection. Nature 2000, 407(6803):523–6. 10.1038/35035103

    Article  Google Scholar 

  24. Kaufmann D, Lichterfeld M, Altfeld M, Allen T, Johnston M, Lee P, Wagner B, Kalife E, Strick D, Rosenberg E, Walker BD: Limited Durability of Immune Control Following Treated Acute HIV Infection. In The 11th Conference on Retroviruses and Opportunistic Infections. San Francisco, CA; 2004.

    Google Scholar 

  25. Frost S, Martinez-Picado J, Ruiz L, Clotet B, Brown A: Viral dynamics during structured treatment interruptions of chronic human immunodeficiency virus type 1 infection. J Virol 2002, 76(3):968–79. 10.1128/JVI.76.3.968-979.2002

    Article  Google Scholar 

  26. Garcia F, Plana M, Ortiz G, Bonhoeffer S, Soriano A, Vidal C, Cruceta A, Arnedo M, Gil C, Pantaleo G, Pumarola T, Gallart T, Nixon D, Miru J, Gatell J: The virological and immunological consequences of structured treatment interruptions in chronic HIV-1 infection. AIDS 2001, 15(9):F29–40. 10.1097/00002030-200106150-00002

    Article  Google Scholar 

  27. Oxenius A, Price D, Gunthard H, Dawson S, Fagard C, Perrin L, Fischer M, Weber R, Plana M, Garcia F, Hirschel B, McLean A, Phillips R: Stimulation of HIV-specific cellular immunity by structured treatment interruption fails to enhance viral control in chronic HIV infection. Proc Natl Acad Sci USA 2002, 99(21):13747–52. 10.1073/pnas.202372199

    Article  Google Scholar 

  28. D'Offzi G, Montesano C, Agrati C, Gioia C, Amicosante M, Topino S, Narciso P, Pucillo L, Ippolito G, Poccia F: Expansion of pre-terminally differentiated CD8 T cells in chronic HIV-positive patients presenting a rapid viral rebound during structured treatment interruption. AIDS 2002, 16(18):2431–8. 10.1097/00002030-200212060-00008

    Article  Google Scholar 

  29. Ortiz G, Wellons M, Brancato J, Vo H, Zinn R, Clarkson D, Loon KV, Bonhoeffer S, Miralles G, Montefiori D, Bartlett J, Nixon D: Structured antiretroviral treatment interruptions in chronically HIV-1-infected subjects. Proc Natl Acad Sci USA 2001, 98(23):13288–93. 10.1073/pnas.221452198

    Article  Google Scholar 

  30. Ruiz L, Martinez-Picado J, Romeu J, Paredes R, Zayat M, Marfil S, Negredo E, Sirera G, Tural C, Clotet B: Structured treatment interruption in chronically HIV-1 infected patients after long-term viral suppression. AIDS 2000, 14(4):397–403. 10.1097/00002030-200003100-00013

    Article  Google Scholar 

  31. Carcelain G, Tubiana R, Samri A, Calvez V, Delaugerre C, Agut H, Katlama C, Autran B: Transient mobilization of human immunodeficiency virus (HIV)-specific CD4 T-helper cells fails to control virus rebounds during intermittent antiretroviral therapy in chronic HIV type 1 infection. J Virol 2001, 75: 234–41. 10.1128/JVI.75.1.234-241.2001

    Article  Google Scholar 

  32. Blankson JN, Gallant JE, Quinn TC, Bartlett JG, Siliciano RF: Loss of HIV-1-specific immunity during treatment interruption in 2 chronically infected patients. JAMA 2002, 288(2):162–4. 10.1001/jama.288.2.162

    Article  Google Scholar 

  33. Fagard C, Oxenius A, Gunthard H, Garcia F, Braz ML, Mestre G, Battegay M, Furrer H, Vernazza P, Bernasconi E, Telenti A, Weber R, Leduc D, Yerly S, Price D, Dawson SJ, Klimkait T, Perneger TV, McLean A, Clotet B, Gatell JM, Perrin L, Plana M, Phillips R, Hirschel B, Study SHC: A prospective trial of structured treatment interruptions in human immunodeficiency virus infection. Arch Intern Med 2003, 163(10):1220–6. 10.1001/archinte.163.10.1220

    Article  Google Scholar 

  34. Douek D, Brenchley J, Betts M, Ambrozak D, Hill B, Okamoto Y, Casazza J, Kuruppu J, Kunstman K, Wolinsky S, Grossman Z, Dybul M, Oxenius A, Price D, Connors M, Koup R: HIV preferentially infects HIV-specific CD4+ T cells. Nature 2002, 417(6884):95–8. 10.1038/417095a

    Article  Google Scholar 

  35. Altfeld M, Rosenberg E, Shankarappa R, Mukherjee J, Hecht F, Eldridge R, Addo M, Poon S, Phillips M, Robbins G, Sax P, Boswell S, Kahn J, Brander C, Goulder P, Levy J, Mullins J, Walker B: Cellular immune responses and viral diversity in individuals treated during acute and early HIV-1 infection. J Exp Med 2001, 193(2):169–80. 10.1084/jem.193.2.169

    Article  Google Scholar 

  36. Palmer B, Boritz E, Blyveis N, Wilson C: Discordance between frequency of human immunodeficiency virus type 1 (HIV-1)-specific gamma interferon-producing CD4(+) T cells and HIV-1-specific lymphoproliferation in HIV-1-infected subjects with active viral replication. J Virol 2002, 76(12):5925–36. 10.1128/JVI.76.12.5925-5936.2002

    Article  Google Scholar 

  37. Ostrowski M, Gu J, Kovacs C, Freedman J, Luscher M, MacDonald K: Quantitative and qualitative assessment of human immunodeficiency virus type 1 (HIV-1)-specific CD4+ T cell immunity to gag in HIV-1-infected individuals with differential disease progression: reciprocal interferon-gamma and interleukin-10 responses. J Infect Dis 2001, 184(10):1268–78. 10.1086/324005

    Article  Google Scholar 

  38. Kaufmann D, Rosenberg E: The value of preserving HIV-specific immune responses. J HIV Ther 2003, 8: 19–25.

    Google Scholar 

  39. Betts M, Ambrozak D, Douek D, Bonhoeffer S, Brenchley J, Casazza J, Koup R, Picker L: Analysis of total human immunodeficiency virus (HIV)-specific CD4(+) and CD8(+) T-cell responses: relationship to viral load in untreated HIV infection. J Virol 2001, 75(24):11983–91. 10.1128/JVI.75.24.11983-11991.2001

    Article  Google Scholar 

  40. Havlir DV: Structured intermittent treatment for HIV disease: Necessary concession or premature compromise? Proc Natl Acad Sci USA 2002, 99: 4–6. 10.1073/pnas.022629399

    Article  Google Scholar 

  41. Ananworanich J, Nuesch R, Braz ML, Chetchotisakd P, Vibhagool A, Wicharuk S, Ruxrungtham K, Furrer H, Cooper D, Hirschel B, Bernasconi E, Cavassini M, Ebnther C, Fagard C, Genne D, Khanna N, Perrin L, Phanupak P, Ubolyam S, Vernazza P, Yerly S, Study SHC: Failures of 1 week on, 1 week off antiretroviral therapies in a randomized trial. AIDS 2003, 17(15):F33–7. 10.1097/00002030-200310170-00001

    Article  Google Scholar 

  42. Abbas UL, Mellors JW: Interruption of antiretroviral therapy to augment immune control of chronic HIV-1 infection: risk without reward. Proc Natl Acad Sci USA 2002, 99(21):13377–8. 10.1073/pnas.212518999

    Article  Google Scholar 

  43. Martinez-Picado J, Morales-Lopetegi K, Wrin T, Prado JG, Frost SDW, Petropoulos CJ, Clotet B, Ruiz L: Selection of drug-resistant HIV-1 mutants in response to repeated structured treatment interruptions. AIDS 2002, 16(6):895–9. 10.1097/00002030-200204120-00009

    Article  Google Scholar 

  44. Luo R, Zurakowski R: A new strategy to decrease risk of resistance emerging during therapy switching in HIV treatment. American Control Conference 2008, 2112–2117.

    Google Scholar 

  45. Luo R, Piovoso M, Zurakowski R: A generalized multi-strain model of HIV evolution with implications for drug-resistance management. American Control Conference, 2009. ACC '09 2009, 2295–2300.

    Chapter  Google Scholar 

  46. Luo R, Piovoso M, Zurakowski R: Modeling-error robustness of a viral-load preconditioning strategy for HIV treatment switching. American Control Conference (ACC), 2010 2010, 5155–5160.

    Google Scholar 

  47. Luo R, Cannon L, Hernandez J, Piovoso MJ, Zurakowski R: Controlling the evolution of resistance. Journal of Process Control 2011, 21(3):367–378. 10.1016/j.jprocont.2010.11.010

    Article  Google Scholar 

  48. Hocqueloux L, Prazuck T, Avettand-Fenoel V, Lafeuillade A, Cardon B, Viard JP, Rouzioux C: Long-term immunovirologic control following antiretroviral therapy interruption in patients treated at the time of primary HIV-1 infection. AIDS 2010, 24(10):1598–1601. 10.1097/QAD.0b013e32833b61ba

    Article  Google Scholar 

  49. Zurakowski R, Teel A: A Model Predictive Control Based Scheduling Method for HIV Therapy. Journal of Theoretical Biology 2006, 238(2):368–82. 10.1016/j.jtbi.2005.05.004

    Article  MathSciNet  Google Scholar 

  50. David J, Tran H, Banks HT: Receding Horizon Control of HIV. Optim Control Appl Meth 2010. n/a-n/a

    Google Scholar 

  51. Pannocchia G, Laurino M, Landi A: A Model Predictive Control Strategy Toward Optimal Structured Treatment Interruptions in Anti-HIV Therapy. Biomedical Engineering, IEEE Transactions on 2010, 57(5):1040–1050.

    Article  Google Scholar 

  52. Landi A, Mazzoldi A, Andreoni C, Bianchi M, Cavallini A, Laurino M, Ricotti L, Iuliano R, Matteoli B, Ceccherininelli L: Modelling and control of HIV dynamics. Computer Methods and Programs in Biomedicine 2008, 89(2):162–168. 10.1016/j.cmpb.2007.08.003

    Article  Google Scholar 

  53. Mhawej MJ, Moog CH, Biafore F, Brunet-François C: Control of the HIV infection and drug dosage. Biomedical Signal Processing and Control 2010, 5: 45–52. 10.1016/j.bspc.2009.05.001

    Article  Google Scholar 

  54. Elaiw AM, Xia X: HIV dynamics: Analysis and robust multirate MPC-based treatment schedules. Journal of Mathematical Analysis and Applications 2009, 359: 285–301. 10.1016/j.jmaa.2009.05.038

    Article  MathSciNet  MATH  Google Scholar 

  55. Chang H, Astolfi A: Enhancement of the immune system in HIV dynamics by output feedback. Automatica 2009, 45(7):1765–1770. 10.1016/j.automatica.2009.03.016

    Article  MathSciNet  MATH  Google Scholar 

  56. Chang H, Astolfi A: Activation of Immune Response in Disease Dynamics via Controlled Drug Scheduling. Automation Science and Engineering, IEEE Transactions on 2009, 6(2):248–255.

    Article  Google Scholar 

  57. Shim H, Jo NH, Chang H, Seo JH: A system theoretic study on a treatment of AIDS patient by achieving long-term non-progressor? Automatica 2009, 45(3):611–622. 10.1016/j.automatica.2008.09.021

    Article  MathSciNet  MATH  Google Scholar 

  58. Chang H, Astolfi A: 2008 47th IEEE Conference on Decision and Control. 2008 47th IEEE Conference on Decision and Control, IEEE 2008, 1759–1764.

    Chapter  Google Scholar 

  59. Kalami Heris S, Khaloozadeh H: Open- and Closed-Loop Multi-Objective Optimal Strategies for HIV Therapy using NSGA-II. Biomedical Engineering, IEEE Transactions on 0 (99):1–1.

  60. Chang H, Astolfi A: Control of HIV Infection Dynamics. Control Systems, IEEE 2008, 28(2):28–39.

    MathSciNet  Google Scholar 

  61. Zurakowski R, Messina M, Tuna S, Teel A: Treatment Scheduling for HIV using nonlinear model predictive control. Australian Journal of Electrical and Electronics Engineering 2005, 2: 49–58.

    Google Scholar 

  62. Wodarz D: Helper-dependent vs. helper-independent CTL responses in HIV infection: implications for drug therapy and resistance. J Theor Biol 2001, 213(3):447–59. 10.1006/jtbi.2001.2426

    Article  Google Scholar 

  63. Bonnabel S, Martin P, Rouchon P: 2006 American Control Conference. American Control Conference, IEEE 2006, 2910–2914.

    Google Scholar 

  64. Mayne D, Rawlings J, Rao C, Scokaert P: Constrained model predictive control: Stability and optimality. Automatica 2000, 36: 789–814. 10.1016/S0005-1098(99)00214-9

    Article  MathSciNet  MATH  Google Scholar 

  65. Grimm G, Messina M, Tuna S, Teel A: Model predictive control: for want of a control lyapunov function, all is not lost. IEEE Trans Automatic Control 2005, 50(5):546–558.

    Article  MathSciNet  Google Scholar 

  66. Messina M, Tuna S, Teel A: Discrete-time certainty equivalence output feedback: allowing discontinuous control laws including those from model predictive control. Automatica 2005, 41(4):617–628. 10.1016/j.automatica.2004.11.015

    Article  MathSciNet  MATH  Google Scholar 

  67. Alessandri A, Baglietto M, Battistelli G, Parisini T: Receding-horizon estimation for noisy nonlinear discrete-time systems. Proceedings of the 42nd IEEE CDC, Maui, Hawaii 2003.

    Google Scholar 

Download references

Acknowledgements

Research supported in part by NIH grant number R21 AI057071. This research was presented in abbreviated form at the American Control Conference.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ryan Zurakowski.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

RZ conceived of the study, designed all methods, implemented all methods, and drafted the manuscript.

Authors’ original submitted files for images

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Zurakowski, R. Nonlinear observer output-feedback MPC treatment scheduling for HIV. BioMed Eng OnLine 10, 40 (2011). https://doi.org/10.1186/1475-925X-10-40

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1475-925X-10-40

Keywords