Skip to main content Accessibility help
×
Home

Higher order symbolic execution for contract verification and refutation*

  • PHÚC C. NGUYÊN (a1), SAM TOBIN-HOCHSTADT (a2) and DAVID VAN HORN (a3)

Abstract

We present a new approach to automated reasoning about higher-order programs by endowing symbolic execution with a notion of higher-order, symbolic values. To validate our approach, we use it to develop and evaluate a system for verifying and refuting behavioral software contracts of components in a functional language, which we call soft contract verification. In doing so, we discover a mutually beneficial relation between behavioral contracts and higher-order symbolic execution. Contracts aid symbolic execution by providing a rich language of specifications serving as a basis of symbolic higher-order values; the theory of blame enables modular verification and leads to the theorem that verified components can't be blamed; and the run-time monitoring of contracts enables soft verification whereby verified and unverified components can safely interact. Conversely, symbolic execution aids contracts by providing compile-time verification and automated test case generation from counter-examples to verification. This relation between symbolic exuection and contracts engenders a virtuous cycle encouraging the gradual use of contracts.

Our approach is able to analyze first-class contracts, recursive data structures, unknown functions, and control-flow-sensitive refinements of values, which are all idiomatic in dynamic languages. It makes effective use of off-the-shelf solvers to decide problems without heavy encodings. Counterexample search is sound and relatively complete with respect to a first-order solver for base type values and counter-examples are reported as concrete values, including functions. Therefore, it can form the basis of automated verification and bug-finding tools for higher-order programs. The approach is competitive with a range of existing tools—including type systems, flow analyzers, and model checkers—on their own benchmarks. We have built a prototype to analyze programs written in Racket and report on its effectiveness in verifying and refuting contracts.

Copyright

Footnotes

Hide All
*

This material is based on research sponsored by the NSF under award 1218390, the NSA under the Science of Security program, and DARPA under the programs Automated Program Analysis for Cybersecurity (FA8750-12-2-0106) and Clean-slate design of Resilient Adaptive Secure Hosts. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon.

Footnotes

References

Hide All
Aiken, A., Wimmers, E. L. & Lakshman, T. K. (1994) Soft typing with conditional types. In Proceedings of the 21st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM - New York, New York, USA, pp. 163–173.
Austin, T. H., Disney, T. & Flanagan, C. (2011, October) Virtual values for language extension. In Proceedings of the 2011 ACM International Conference on Object Oriented Programming Systems Languages and Applications. ACM, pp. 921–938.
Barrett, C., Conway, C., Deters, Hadarean L., Jovanović, D., King, T., Reynolds, A. & Tinelli, C. (2011) Cvc4. In International Conference on Computer-Aided Verification. Springer, pp. 171177.
Cadar, C., Dunbar, D. & Engler, D. (2008) KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation. USENIX Association, pp. 209–224.
Cadar, C., Ganesh, V., Pawlowski, P. M., Dill, D. L. & Engler, D. R. (2006) EXE: Automatically generating inputs of death. In Proceedings of the 13th ACM Conference on Computer and Communications Security. ACM, pp. 322–335.
Cartwright, R. & Fagan, M. (1991) Soft typing. In Proceedings of the ACM SIGPLAN 1991 Conference on Programming Language Design and Implementation. ACM, pp. 278–292.
Cartwright, R. & Felleisen, M. (1996, June) Program verification through soft typing. ACM Comput. Surv. 28 (2) 349351.
Chugh, R., Herman, D. & Jhala, R. (2012a, October) Dependent types for JavaScript. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications. ACM, pp. 587–606.
Chugh, R., Rondon, P. M. & Jhala, R. (2012b, January) Nested refinements: A logic for duck typing. In Proceedings of the 39th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, pp. 231–244.
Claessen, K. & Hughes, J. (2000) QuickCheck: A lightweight tool for random testing of Haskell programs. In Proceedings of the Fifth ACM SIGPLAN International Conference on Functional Programming. ACM, pp. 268–279.
Dimoulas, C., Findler, R. B., Flanagan, C. & Felleisen, M. (2011, January) Correct blame for contracts: No more scapegoating. In Proceedings of the 38th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, pp. 215–226.
Disney, T. (2013, July) contracts.coffee.
Disney, T., Flanagan, C. & McCarthy, J. (2011, September) Temporal higher-order contracts. In ICFP '11 Proceeding of the 16th ACM SIGPLAN International Conference on Functional Programming. ACM, pp. 176–188.
Fähndrich, M. & Logozzo, F. (2011) Static contract checking with abstract interpretation. In Proceedings of the 2010 International Conference on Formal Verification of Object-Oriented Software. Springer, pp. 10–30.
Findler, R. B. & Felleisen, M. (2002, September) Contracts for higher-order functions. In ICFP '02: Proceedings of the seventh ACM SIGPLAN International Conference on Functional Programming. ACM, pp. 48–59.
Flanagan, C. & Felleisen, M. (1999, March) Componential set-based analysis. ACM Trans. Program. Lang. Syst. 21 (2) 370416.
Flanagan, C., Flatt, M., Krishnamurthi, S., Weirich, S. & Felleisen, M. (1996, May) Catching bugs in the web of program invariants. In PLDI '96: Proceedings of the ACM SIGPLAN 1996 Conference on Programming Language Design and Implementation. ACM, pp. 23–32.
Flatt, M. & PLT (2010) Reference: Racket. Technical Report PLT-TR-2010-1, PLT Inc.
Foster, J. S., Terauchi, T. & Aiken, A. (2002, May) Flow-sensitive type qualifiers. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, pp. 1–12.
Freeman, T. & Pfenning, F. (1991, June) Refinement types for ML. In Proceedings of the ACM SIGPLAN 1991 Conference on Programming Language Design and Implementation. ACM, pp. 268–277.
Godefroid, P., Klarlund, N. & Sen, K. (2005, June) DART: Directed automated random testing. In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, pp. 213–223.
Greenberg, M., Pierce, B. C. & Weirich, S. (2010) Contracts made manifest. In POPL '10: Proceedings of the 37th annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, pp. 353–364.
Heidegger, P. & Thiemann, P. (2010) Contract-Driven testing of JavaScript code. In Objects, Models, Components, Patterns. Berlin Heidelberg: Springer, pp. 154172.
Henglein, F. (1994, June) Dynamic typing: Syntax and proof theory. Sci. Comput. Program. 22 (3) 197230. Elsevier, Amsterdam, Netherlands.
Hickey, R., Fogus, M. & contributors (2013, July). core.contracts.
Johnson, J. I. & Van Horn, D. (2014, October) Abstracting abstract control. In Proceedings of the 10th ACM Symposium on Dynamic Languages. ACM, pp. 11–22.
Kawaguchi, M., Rondon, P. & Jhala, R. (2010) Dsolve: Safety verification via liquid types. In Computer Aided Verification. Berlin Heidelberg: Springer, pp. 123126.
Klein, C., Flatt, M. & Findler, R. B. (2010) Random testing for higher-order, stateful programs. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications. ACM, pp. 555–566.
Knowles, K. & Flanagan, C. (2010, February) Hybrid type checking. ACM Trans. Program. Lang. Syst. 32 (2) article 6.
Kobayashi, N. (2009a) Model-checking higher-order functions. In Proceedings of the 11th ACM SIGPLAN Conference on Principles and Practice of Declarative Programming. ACM, pp. 25–36.
Kobayashi, N. (2009b, January) Types and higher-order recursion schemes for verification of higher-order programs. In Proceedings of the 36th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, pp. 416–428.
Kobayashi, N. & Igarashi, A. (2013) Model-Checking Higher-Order programs with recursive types. In European Symposium on Programming. Berlin Heidelberg: Springer, pp. 431450.
Kobayashi, N. & Ong, C. H. L. (2009, August) A type system equivalent to the modal Mu-Calculus model checking of Higher-Order recursion schemes. In Proceedings of the 24th Annual IEEE Symposium on Logic In Computer Science. IEEE, pp. 179–188.
Kobayashi, N., Sato, R. & Unno, H. (2011, June) Predicate abstraction and CEGAR for higher-order model checking. In Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, pp. 222–233.
Kobayashi, N., Tabuchi, N. & Unno, H. (2010, January) Higher-order multi-parameter tree transducers and recursion schemes for program verification. In Proceedings of the 37th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, pp. 495–508.
Meunier, P., Findler, R. B. & Felleisen, M. (2006, January) Modular set-based analysis from contracts. In POPL '06: Conference Record of the 33rd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, pp. 218–231.
Meyer, B. (1991, October) Eiffel: The Language. Prentice Hall.
Moura, L. D. & Bjørner, N. (2008) Z3: An efficient SMT solver. In Proceedings of the Theory and Practice of Software, 14th International Conference on Tools and Algorithms for the Construction and Analysis of Systems. Springer-Verlag, pp. 337–340.
Nguyên, P. C., Tobin-Hochstadt, S. & Van Horn, D. (2014) Soft contract verification. In Proceedings of the 19th ACM SIGPLAN International Conference on Functional Programming. ACM, pp. 139–152.
Nguyên, P. C. & Van Horn, D. (2015) Relatively complete counterexamples for higher-order programs. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, pp. 446–456.
Ong, C. H. L. (2006) On Model-Checking trees generated by Higher-Order recursion schemes. In 21st Annual IEEE Symposium on Logic in Computer Science (LICS'06). IEEE, pp. 81–90.
Ong, C. H. L. & Ramsay, S. J. (2011, January) Verifying higher-order functional programs with pattern-matching algebraic data types. In Proceedings of the 38th annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, pp. 587–598.
Plosch, R. (1997, December) Design by contract for Python. In Proceedings of the Joint Asia Pacific Software Engineering Conference. IEEE, pp. 213–219.
Rondon, P. M., Kawaguci, M. & Jhala, R. (2008) Liquid types. In Proceedings of the 2008 ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, pp. 159–169.
Seidel, E., Vazou, N. & Jhala, R. (2015) Type targeted testing. In Programming Languages and Systems. Berlin Heidelberg: Springer, pp. 812836.
Shivers, O. (1988) Control flow analysis in Scheme. In PLDI '88: Proceedings of the ACM SIGPLAN 1988 Conference on Programming Language Design and Implementation. ACM, pp. 164–174.
St-Amour, V., Tobin-Hochstadt, S., Flatt, M. & Felleisen, M. (2012) Typing the numeric tower. In Practical Aspects of Declarative Languages. Berlin Heidelberg, Springer, pp. 289303.
Strickland, T. S., Tobin-Hochstadt, S., Findler, R. B. & Flatt, M. (2012, October) Chaperones and impersonators: Run-time support for reasonable interposition. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications. ACM, pp. 943–962.
Terauchi, T. (2010) Dependent types from counterexamples. In Proceedings of the 37th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, pp. 119–130.
Tobin-Hochstadt, S. & Felleisen, M. (2010, September) Logical types for untyped languages. In ICFP '10: International Conference on Functional Programming. ACM, pp. 117–128.
Tobin-Hochstadt, S., St-Amour, V., Culpepper, R., Flatt, M. & Felleisen, M. (2011, June) Languages as libraries. In Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, pp. 132–141.
Tobin-Hochstadt, S. & Van Horn, D. (2012) Higher-order symbolic execution via contracts. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications. ACM, pp. 537–554.
Tsukada, T. & Kobayashi, N. (2010) Untyped recursion schemes and infinite intersection types. In Proceedings of the 13th International Conference on Foundations of Software Science and Computational Structures. Springer-Verlag, pp. 343–357.
Van Horn, D. & Might, M. (2010, September) Abstracting abstract machines. In Proceedings of the 15th ACM SIGPLAN International Conference on Functional Programming. ACM, pp. 51–62.
Van Horn, D. & Might, M. (2012) Systematic abstraction of abstract machines. Journal of Functional Programming 22(Special Issue 4–5) Cambridge University Press, 705746.
Vazou, N., Rondon, P. & Jhala, R. (2013) Abstract refinement types. In European Symposium on Programming. Berlin Heidelberg: Springer, pp. 209228.
Vazou, N., Seidel, E. L., Jhala, R., Vytiniotis, D. & Jones, S. P. (2014, August) Refinement types for Haskell. In Proceedings of the 19th ACM SIGPLAN International Conference on Functional Programming. ACM, pp. 269–282.
Vytiniotis, D., Jones, S. P., Claessen, K. & Rosén, D. (2013) Halo: Haskell to logic through denotational semantics. In Proceedings of the 40th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, pp. 431–442.
Wright, A. K. & Cartwright, R. (1997, January) A practical soft type system for Scheme. ACM Trans. Program. Lang. Syst. 19 (1) ACM, 87152.
Xie, Y. & Aiken, A. (2005, January) Scalable error detection using boolean satisfiability. In Proceedings of the 32nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, pp. 351–363.
Xu, D. N. (2012) Hybrid contract checking via symbolic simplification. In Proceedings of the ACM SIGPLAN 2012 Workshop on Partial Evaluation and Program Manipulation. ACM, pp. 107–116.
Xu, D. N., Peyton Jones, S. & Claessen, S. (2009) Static contract checking for Haskell. In POPL '09: Proceedings of the 36th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, pp. 41–52.
Yang, J., Twohey, P., Engler, D. & Musuvathi, M. (2004) Using model checking to find serious file system errors. In Proceedings of the 6th Symposium on Operating Systems Design and Implementation. USENIX, pp. 273–287.
Zhu, H. & Jagannathan, S. (2013) Compositional and lightweight dependent type inference for ML. In Conference on Verification, Model-Checking and Abstract Interpretation Springer, 295–314.

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed

Higher order symbolic execution for contract verification and refutation*

  • PHÚC C. NGUYÊN (a1), SAM TOBIN-HOCHSTADT (a2) and DAVID VAN HORN (a3)
Submit a response

Discussions

No Discussions have been published for this article.

×

Reply to: Submit a response


Your details


Conflicting interests

Do you have any conflicting interests? *