One approach to agent-based social system modeling is the Institutional Analysis and Development Framework developed by Elinor and Vincent Ostrom at the Indiana University:

The IAD Framework offers researchers a way to understand the policy process by outlining a systematic approach for analyzing institutions that govern action and outcomes within collective action arrangements (Ostrom, 2007, 44). Institutions are defined within the IAD Framework as a set of prescriptions and constraints that humans use to organize all forms of repetitive and structured interactions (Ostrom, 2005, 3).  These prescriptions can include rules, norms, and shared strategies (Crawford and Ostrom 1995; Ostrom 1997). Institutions are further delineated as being formal or informal; the former characterized as rules-in-form and the latter as rules-in-use.

The IAD framework identifies key variables that researchers should use in evaluating the role of institutions in shaping social interactions and decision-making processes.  The analytical focus of the IAD is on an “action arena”, where social choices and decisions take place. Three broad categories of variables are identified as influencing the action arena:  institutions or rules that govern the action arena, the characteristics of the community or collective unit of interest, and the attributes of the physical environment within which the community acts (Ostrom 1999; Ostrom 2005). Each of these three categories has been further delineated by IAD scholars into relevant variables and conditions that can influence choices in the action arena.  For instance, the types of rules that are important in the IAD include entry and exit rules, position rules, scope rules, payoff rules, aggregation rules, authority rules, and information rules.  Key characteristics of the community can include factors such as the homogeneity of its members or shared values.  Biophysical variables might include factors such as the mobility and flow of resources within an action arena.

The IAD further defines the key features of “action situations” and “actors” that make up the action arena. The action situation has seven key components: 1) the participants in the situation, 2) the participants’ positions, 3) the outcomes of participants’ decisions, 4) the payoffs or costs and benefits associated with outcomes, 5) the linkages between actions and outcomes, 6) the participants’ control in the situation, and 7) information. The variables that are essential to evaluating actors in the action arena are 1) their information processing capabilities, 2) their preferences or values for different actions, 3) their resources, and 4) the processes they use for choosing actions.

Here are a couple papers that describe attempts to operationalize this framework:

MAIA: a Framework for Developing Agent-Based Social Simulations

Modelling socio-ecological systems with MAIA: A biogas infrastructure simulation