Sustainable ICT Holdings is an active shareholder of Good Sleep Pty Ltd, and continues to contribute time and money to The Good Sleep startup. The Good Sleep Toolkit is an innovative mobile technology service for reducing the clinical pathway for constituents with sleep health issues. The solution reduces the total cost of clinical services to the patient, and improves General Practitioner efficacy when assessing patients with sleep concerns.

http://www.goodsleeptoolkit.com

Whilst everyone is talking about AI and what it can do, there is very little out there that deals with how it can be adopted within a business.

What does it take to implement, and what do I need to consider from a development and management perspective?

With today’s AI-hype, one could presume that the development of Artificial Intelligence or Machine Learning is as easy as it is to develop a website today.

Salesforce.com has released Einstein, through acquisition, and is obviously positioning its offering within the context of CRM data. Though Einstein can only address certain problems at the moment, such as image recognition and customer service “bots”. It cannot be easily adapted to create a whole new model such as evaluating the validity of an insurance claim submitted via Salesforce Communities for example. It requires sophisticated development. I’m sure Salesforce’s research teams will be working hard to address this.

Google Cloud Platform, Amazon AWS, and Microsoft Azure all have Machine Learning compute services, which are absolutely necessary due to the GPU and Memory requirements of AI and ML, but again this relies on your business having the capacity to choose and develop the appropriate Machine Learning or Deep Learning models and then releasing them to these environments. Which highlights a good point that if your organisation hasn’t adopted a cloud compute strategy, you will likely be at a disadvantage when it comes to AI and ML development.

Frameworks have taken up a lot of the heavy lifting. Tensorflow, Keras, Caffe, Torch et al. are all a “god-send” for development, and each has its strengths (link). But they are just frameworks, which abstract away the complexity of the underlying mathematics that powers AI and ML (see matrix mathematics and back-propogation as examples), they still require considerable data processing and coding.

So it is unlikely that the future will see any of the aforementioned vendors release a simple and intuitive solution for general Deep Learning and Machine Learning development. Therefore its best to understand how to approach this within our organisations.

There are four important aspects that need careful planning and execution. Failure to undertake such care will lead to either failing to deliver the solution, or worse still, a solution that over time fails to meet the accuracy expected. The latter being disastrous in some scenarios. So here are the areas;

A. Understand your “why”
Foremost in the process is being very clear about the Question we are seeking to answer with Machine Learning or our Deep Learning implementation. The right question will drive data selection, algorithm and framework best for your environment.

B. Understand your Data Sources
In seeking to answer our question, the identification of data sources can be a lengthy process. It requires identifying our source data, shaping the data to suit our model type, and validating our data. If you’re interested in learning more about shaping data for this purpose, a good start is Tidy Data by Hadley Wickham that a colleague of mine put me onto.
Companies that already have a mature data pipeline and BI environment will be part the way there. Data pre-processing is a skill and requires a good toolkit. In future I predict you will see companies such as Zoomdata building real-time data pipelines specifically to support Deep Learning implementations – watch this space.

C. Correct Algorithm Selection
Not all algorithms are created equal, and there are a plethora of algorithms (here, here) that are best for different problem domains. This is where a sizable amount of time must be invested. It will determine whether Deep Learning algorithms (ReLU, Softmax or both etc), or Machine Learning (KNN, SVM, or K-Means clustering etc) algorithms are appropriate, each has its place. Choose the wrong one and you’ll have sub-optimal results

D. Integrating into existing methods
As with any number of scientific endeavours Deep Learning and Machine Learning requires a combination of Science, Experience based Intuition and Art. Estimating time to develop models will be difficult depending on the type of Question being asked (see above) and the experience of the team. On this basis it is well suited to rapid test and release, as prescribed by Agile methods.

A method for implementation

With these aspects of Deep Learning or Machine Learning adoption in mind, applying them in an agile method will require four (4) key steps (Figure 1.a)

The method is triggered by an area of enquiry (our “why”), based on the a particular business problem and available data.

Acquire Stage: The formal identification of data sources, both public and private that can be used in the implementation. With most implementations, you need a significant amount of data to Train, Test and Validate our model. Defining the sourcing strategy, for the initial development and incremental training once complete will be necessary.

Process & Transform Stage: Entails data pre-processing of data sources and shaping data. This will result in identifying the required features necessary to train our model and normalising our values. The output of this stage is essential for evaluating our approach.

Qualify & Evaluate: This stage requires that we evaluate our input data, remove outliers that will impact our accuracy, and determine which algorithms and algorithm/s is best to answer our question. Visualising our data is essential. Note there is often multiple algorithms used depending on your solutions. For example, Deep learning will use different algorithms in layers of our model. A team may spend a large amount of time to evaluate different algorithms, so patience is required.

Train & Validate: The development of the model with the approach decided is undertaken in this stage, using our separated datasets and we evaluate the predictions accuracy. We choose our hyper-parameters, and adjust based on training results. Training a model can take time, and often requires significant intuition as currently its largely a manual undertaking, though the actual coding effort is generally not large when you have an experienced practitioner.

The investment required in execution leans heavily towards the design stage (Figure 1.b above). As we iterate through the phases we reduce our uncertainty and our pace of execution will increase. Once we have a production solution, the pipeline to retrain the model is already build and therefore should be largely automated. Yes, I did say retrain the model – as you gather more data your need to retrain to improve prediction accuracy depending on the type of algorithms used.

It’s an exciting endeavour, and one that can drive enormous benefits if you appreciate the steps to get there. It is still an emerging field, and there are limited toolsets that make development simple, but with the right team you can get the results.

• Lack of organizational ability to quantifiably define and measure EA Practice performance
• Failure to appropriately adapt a proven Enterprise Architecture Framework to the organization
• Adoption of frameworks that do not sufficiently address the interfaces with Architecture Delivery
• Inexperience of practitioners in architecture delivery
• EA practitioners who have spent too much time abstracted from the reality of architecture delivery.

• When was the last time you sat beside an Architecture Delivery project team for an extended period of time and actually mopped up the proverbial blood with Solution Architects?
  It is only through spending time embedded in Architecture Delivery roles and activities that we can maintain relevance to a field of experienced professionals who have dedicated their careers to delivering outcomes that are unquestionably measurable. It is a mistake to think that as Enterprise Architects we are more experienced and knowledgeable than a Solution Architect, because the disciplines require different types of experiences.
• Have you established the correct team composition for your Enterprise Architecture practice? Have you covered the bases necessary for your Architecture Plan?
  Diversity of experience and specialization is important, but so are high levels of emotional intelligence. A team lacking the ability to develop personal relationships and read non-verbal signs of stakeholders and project delivery teams is limited in its ability to identify and address issues early. It should no longer be acceptable to dispense with our responsibility for seeing the plan become realized for the enterprise; and close working relationships are critical to doing so. If the team composition is not right to realize the plan, it needs changing.

• What physical architecture artefacts (models, patterns, standards, etc.) have you published in a way that can be effectively reused by the Solution Architect?
  Heading into 2014, it is surprising to find many large organizations that have failed to establish a formal, structured, repository for architecture storage, management, interrogation, and retention. An architecture that cannot be reused is ultimately devalued.
• What tooling have you made available for the management, discovery, and reuse of architectures?
  The continued exclusive use of Microsoft Visio or, worse still, a variety of non-integrated tools depending on the type of Architecture being developed, will continue to degrade the value of enterprise architectures. Do boxers continue to use the same old boxing gloves and training methods of the days of old, or do they adopt new proven glove technology and training approaches to maximize the opportunity to be successful?
• Have you formalized an Architecture Delivery framework and/or standard?
  Regardless of how implementation is undertaken, without a defined process and templates for Architecture Delivery that ensure completeness, consistency, and domain coverage, Solution Architecture will be less than efficient or lack demonstrable alignment to the Architecture Plan. Like Architecture Planning frameworks such as TOGAF and FEAF, there are Architecture Delivery frameworks such as DoDAF and/or methods and standards such as 4+1 and ISO/IEC 42010 that provide guidance and rules for defining, structuring, classifying, and organizing architectures.

• What processes have you implemented as a formal method for incorporating change into published strategies, standards, and architecture models based on Solution Architecture feedback?
  This is more than communities of practice and weekly conference calls. Often there is too much talk and not enough action. Structured and actionable methods of change initiation, review, and approval need to be implemented and, of course, measured. Otherwise Architectures become stagnant.
• What processes or practices have you implemented to identify, rapidly respond, and support Solution Architects when risks and issues are raised?
  How often do you review the risk and issues log of projects and call the Solution Architect and enquire about delivery progress. The difference lays in how active and engaged we are in the realization of the Architecture Plan. Personal engagement and commitment to the success of a program/project builds an Enterprise Architects credibility.

I am currently engaged in some research on an approach towards quantifying organisational capabilities. Through this process I was reading ISO31000: 2009 (Risk Management Standard) from the International Standards Organisation (ISO) that succinctly defined “risk” as;

This is particularly appropriate within the context of defining Architectures. I’d go so far as to assert that reducing uncertainty is one of the fundamental axioms of architecture, and therefore the role of an architect. Some simple examples to explain;

• A Solution Architecture will reduce business risk relating to a solutions ability meet particular requirements (objectives). Said requirements, typically defined across business, functional and non-functional aspects of a solution, collectively deliver to an organisational outcome.  The architecture will reduce uncertainty regarding business interfaces, deployment and necessary support structures.
• A Technology Strategy, and accompanying road-map, reduces uncertainty relating to required technology investment for ICT capabilities necessary to support the business objectives over a prescribed planning horizon. Therefore, such planning reduces organisational risk that ICT investment is ill-directed.
• A Segment Architecture for Integrated Customer Service will reduce executive uncertainty regarding the level of ICT systems and organisational change required to support a new customer service division. The endorsed segment architecture will improve the organisations chances for success before venturing into a change program and therefore reduce the risk of abject failure.

Therefore it is a reasonable assertion that through the minimization of uncertainty, relating to an architectures ability to meet business objectives, the Architecture Practitioner is fundamentally managing and reducing risk. 

If you accept this axiom as true, you then begin to question the application of risk management principles to the way in which architectures are created. Through the adoption of a risk driven approach, an Architecture practitioner can address organisational concerns by focusing on components of the solution prioritized through risk assessment criteria. 

Every organisation will have their own risk appetite; The level of risk they are happy to carry through different types of engagements. The lower the risk appetite the more complete the architecture needs to be, and of course the inverse therefore applies. A simple risk matrix with analysis axis included below by way of example. As in Figure 1, where the likelyhood is low, and the impact is negligible, only light-touch analysis and design would be required. However if a component of the architecture has a major impact, and is likely to occur, then considerably more analysis and design will be required.

To build on this concept, an “Acceptable Risk Threshold” would represent an organisations risk appetite, and would be used to guage the level of analysis necessary within the architecture scope. As with Figure 1, the risk appetite is quite balanced, where only components of an architecture which are likely-or-almost certain, and a major-to-severe impact requiring the most analysis. Graphically this can be viewed in Figures 2 and 3, which show organisations with different Acceptable Risk Thresholds. Hopefully you can tell, from the images, that Figure 2 represents an organisation that has a low risk threshold (4). Figure 3 however is more middle level and somewhat reflective of the risk threshold in Figure 1 above with a higher risk threshold (11).

Such an approach is a considerable departure from standard practices. To support such an approach however would require an organisation to be quite structured in their architecture practice, and requires additional upfront planning and analysis in risk assessment of the components that make up the architecture. Standardized views and viewpoints, modelling notation and analysis methods would need to be in place, and the risk threshold could then be applied to the development of Statements of Architecture Work. Governance bodies would perhaps have a better means through which to assess completion of the Architecture, rather than a mere “tick-n-flick” that a document has been completed.

Undertaking Architecture itself will inevitably reduce implementation risk; though perhaps through the adoption of a Risk Driven Design approach, an Architecture will also be more likely to address the organisations needs through focusing effort on those areas that represent the greatest risk, rather than creating overly burdensome Architectural deliverable’s. Sure, the concept itself requires further development to prove within a working environment, however the basis behind the approach would seem to warrant further exploration.

There are many instances that I can recall of organisations where the scope of in-flight projects were adjusted due to governance decisions with a focus on schedule and cost. In such instances, often features of longer term enterprise architecture importance are de-prioritised in favour of achieving shorter term budgetary and schedule imperatives. Capabilities such as systems integration and messaging often feature particularly high in examples of such de-scoping fervour. On occasion decisions have a dependency impact on other in-flight initiates that are too often inadequately considered in the decision making process. The implications of governance decisions impact Enterprise Architects and Solution Architects alike, and strategies to combat potentially problematic decisions are invaluable.

By way of example, sometime ago I was requested to assist two large programs of work delivering distinctly separate line of business applications. Each program of work was a multi-year and multi-million committed by the enterprise. Whilst the business case and initial scope of work for each program had identified bi-directional systems integration as a significant enterprise benefit, a lack of strong stakeholder support resulted in each program of work de-scoping integration due to schedule and budgetary pressures.

With scope adjusted the architects for the solutions did not consider integration concerns within the architecture. In fact any consideration to integration was so minimal that in future it would eventually turn out that there were significant limitations in the level of integration that could be achieved in one of the platforms. Significant business benefits were at risk of not being realised.

This is just one example where program governance decisions made during program delivery have directly impacted the longer term enterprise architecture vision. Whilst I won’t go into detail of the activities required to take corrective action for the programs in my example, or the political turmoil that ensued, it brings to light shortfalls in the governance frameworks adopted in such organisations.

Whilst the difference between Architecture Governance and Program Governance in large programs of work is a topic for a separate discussion, for this post however I’m more interested in exploring what I believe individual practitioners can do to mitigate deficiencies in governance decision making.

As individual architecture practitioners, it is unrealistic to think we can always influence governance decision making to favour architectural outcomes. You win some, and you lose some, however we can consider some of the following practices as mechanisms to protect the architectural integrity in the interests of the enterprise vision;

“Bottom-up and Top-down” awareness
First learn the meaning of what you say, and then speak – Epictetus

Today more than ever, it is essential to remind ourselves that as Architecture practitioners we all play a part in delivering to an enterprise architecture, regardless of role and position description. Therefore practitioners need to be conversant and supportive of each architecture discipline and its purpose within an enterprise. This is necessary as markets become more dynamic and enterprises seek to become more responsive or ‘agile’.

Program Architects and Solution Architects need to be conversant in enterprise architecture parlance, the framework adopted within the enterprise and the published Enterprise Architecture artifacts such as road-maps, capability maps and motivation models etc. Armed with this knowledge, the Architect will be more prepared and capable of challenging potential Program/Project scope changes that could impact the integrity of the enterprise architecture itself.

On the flip side, Enterprise Architects should be well-informed of the business, technical and governance challenges that will exist in the delivery of solutions that have been scoped as part of the Enterprise Architecture planning process. Through the formulation of the roadmap and scoping of required program/s of work, the Enterprise Architect, or nominated Program Architect, must ensure that dependencies are identified and risks associated with delivery are acknowledged early. This not only provides a more considered foundation for a Solution Architect to begin with, but also ensures that downstream impacts of the program delivery structure have been considered. Deficiencies in the aforementioned knowledge domains and a lack of investment upfront in program of work planning, especially in the business and technical dimensions, will limit the Enterprise Architects ability to advise and support architecture practitioners when confronted with problematic governance decisions.

Architecture Scope IS NOT Program/Project Scope

Whether a practitioner is engaged as an Enterprise Architect, Program Architect, or Solution Architect, his/her initiative is typically bound within the scope of a particular planned initiative. However the initiative scope should not be the sole basis for the scope of architecture analysis. This assertion, I believe, applies to all architecture disciplines.

The program/project scope is, typically, defined along Schedule, Cost and Quality dimensions (I’ve simplified here in the interests of brevity). Each of these aspects of a program/project scope are important to manage the delivery of the end solution, however I’m yet to see any program/project management plan or statement of work that includes “realising enterprise architecture outcomes” as a project objective. This is why the Architecture practitioner is typically responsible for ensuring the ongoing alignment of the solution with the enterprise architecture.

Take the Enterprise Architect required to define a Segment Architecture for a new Service Management business unit (example figure below) which includes business services that are consumed by external stakeholder groups (customers) and interacts with a number of internal services to deliver said services.

The definition of said architecture would be undertaken within a defined engagement scope or conceivably encapsulated within a broader strategic initiative. However should the desired internal services be revised to a subset of those originally intended, the architecture should continue to consider the de-scoped services regardless. The extent to which the architecture analyses and defines these services may be constrained to Use Cases only, for example, depending on the level of risk it represents to realising the longer-term business outcomes. Without any type of analysis, the architecture defined could limit the enterprises ability to eventually address the previously de-scope internal services or, worse still, lead the enterprise into a whole new change program.

Similarly, a Solution Architect required to define an Architecture for a new line of business application would be engaged within the scope of an approved delivery project. In the event the project scope is revised to exclude important functions that have a broader strategic benefit, the Solution Architect should continue to include these functions within analysis and definition activities. Again, consideration of the functions may be at the Use Case level, or even System and Sub-system context. Either way a level of analysis that is commensurate to the strategic importance of the function should be undertaken.

Through accepting that an architecture should not be explicitly constrained by program/project scoping dimensions, practitioners can produce architectures of greater enterprise value and reduce potential for the re-occurrence of examples like the one I provided at the start of my post.

Understand the political landscape

Within any delivery program/project, the architecture practitioner is critical to successful delivery through the removal of uncertainty as it relates to Business, Information, Application and Technical dimensions. Understanding the political landscape inside and outside the program/project in which the practitioner is engaged is important to understanding how to influence governance decision-making.

Practitioners need to work as peers with Program/Project managers, regardless of the project or business hierarchy. An Architect needs to have the courage to challenge decisions that may impact the ability of the architecture to deliver to business outcomes. This will lead to a tension between the roles, however this should be viewed as natural given the distinctly different focuses of the disciplines. An architecture practitioner who understands such a working relationship is more empowered to challenge decisions through appropriate channels, not just operate as a subordinate to a Program/Project Manager.

Similarly, close relationships with business stakeholders will help influencing decision makers. This is not just IT stakeholders, but those who will be the consumers of the solution itself. In some instances access to stakeholders may be limited based on program/project structure, however you should take any opportunity you can to access said stakeholders and become known to them. Having close relationships with business stakeholders is critical to being able to influence governance decision-making.

Finally, with the aforementioned knowledge of enterprise architecture practice, start leveraging the Enterprise Architecture practice in the way it was intended. Establishing a close working relationship with the Enterprise Architect who is responsible for the domain in which you are working is the first step. Engage them directly, regularly and actively regarding architecture scope, risks and issues. An engaged Enterprise Architect can assist in opening doors to stakeholder groups and tackling governance decisions that may impact the enterprise architecture vision.

Architecture Practitioners of all levels and experience can support the delivery of successful business outcomes greater than the development of architecture deliverables constrained by the initiative scope. As practitioners we must do more to deliver an architecture that is aligned with an enterprise vision and should engage outside of a project construct to be adequately prepared to challenge decisions that may be an impediment to this objective.