Data about customers, transactions, locations, assets, processes, and service offerings are scattered across various business units, databases, and instances of internal and external applications. This fragmentation poses a challenge when other systems, processes, and data scientists need to access the data for analysis and AI implementation.

Two fundamental approaches can address this issue: a centralised ontology and decentralised, proto-based, service-oriented architecture.

Centralised ontology systems, such as Palantir Foundry or ETL processes that culminate in a central data warehouse, are designed to provide a centralised repository of metadata and data integration services. They are typically used in large organisations where data is stored in various disparate systems and need to be integrated and analysed. These systems provide a way to define common data models and integrate data from multiple sources.

Consider centralised ontologies as the "one schema to rule them all" - a meta-database that is the canonical truth into and through which all data must flow. Like the CCP, these systems solve integration challenges with an authoritarian imposition of order, structure and ideological dogma. The advantage of this approach is that disputes are arbitrated centrally, allowing strict control over data governance and security. This centralisation, however, comes at the expense of flexibility and speed of innovation and can make these architectures costly to develop and maintain.

Service Oriented Architectures, popularised by Amazon and Netflix, on the other hand, are like Switzerland. Each Canton has complete autonomy but has a strict contract in how it interfaces with the rest of the country. In the case of a Service Oriented Architecture, each system must expose its data and services via a well-defined set of APIs - in effect, a contract with the rest of the enterprise. This approach allows for a more flexible and adaptable architecture that can be easily extended to support new data sources and services.

Protocol buffers (Protos) are often used to write this contract. Let's use an example from Netflix's world. You can read more on their engineering blog: https://netflixtechblog.com/tagged/protocol-buffers

At Netflix, the team responsible for managing the process of making a movie defines a resource object called Production that contains all the information relevant to that movie as it moves through the production process. Some of that data, however, needs to come from other teams. The scripts team supports that integration by providing that data via a clean and contractually defined API.

// Contains Production-related information  
// Note: in the film and TV industry, the term production refers to the process of making a movie, not the environment to run a software
message Production {
  string id = 1;
  string title = 2;
  ProductionFormat format = 3;
  repeated CastMember cast_members = 4; // Data coming from Scripts team
  ProductionSchedule schedule = 5;
  // ... more fields
}

// The Script & Casting team has independently built and deployed a Scripts service that other teams can use to integrate script data
service ScriptsService {
  // Create a report template method.
    rpc GetScript (GetScriptRequest) returns (Script) {}
}

// The scripts team has a well defined contract for their Script resource that also defines the cast
message Script {
  string id = 1;
  repeated CastMember cast_members = 2;
  message CastMember {
    string character = 1;
    string actor = 2;
  }
}

Proto-based service-oriented architectures are designed to provide a more decentralised approach to data integration. They use a protocol buffer (Proto) schema to define data models, which can then be used to generate code for client and server applications. This approach allows for a more flexible and adaptable architecture that can be easily extended to support new data sources and services.

A proto-based service-oriented architecture offers flexibility, scalability, autonomy, and support for cross-functional teams due to its schema-driven approach and loose coupling of services. This allows for rapid adaptation to changing business needs, independent development and deployment of services in various languages, and improved collaboration among different functional areas. The architecture also enhances scalability, performance, fault tolerance, and error handling by enabling horizontal scaling across multiple servers or cloud instances, graceful error management, and independent service monitoring. Lastly, it ensures better security and data governance through the implementation of security protocols, access controls, and distributed ledger management for greater transparency and accountability.

At Alis Exchange, we've experienced how adopting a protobuf-based Service Oriented Architecture unlocks speed and innovation. We developed our Alis Build platform to make this approach to solving the integration process as simple and accessible as possible.

The platform makes it easy for developers, actuaries, business analysts and data scientists to rapidly Define high-quality API contracts, Build the underlying service and business logic and Deploy these services to the cloud.

• Lead Time: The time from code committed to running in production;
• Deployment Frequency: How often deploys happen;
• Mean Time To Restore (MTTR): How quickly can teams restore service after production outages; and
• Change Fail Rate: What percentage of deploys result in service impairment or an outage.

In their research they found that high performing teams and organisations operated significantly faster than their peers. These high performers were able to deploy to production in hours, rather than months, and deploy multiple times per day without compromising stability or uptime.

Developer experience plays an essential enabling role in achieving this level of performance. In our experience in working with builders and digital innovators, we would suggest another metric for developer experience excellence: Time-to-usefulness.

Time-to-usefulness

We define time-to-usefulness (TTU) as:

The time it take a newcomer to transition from 'new laptop' to 'business impact'.

Newcomers may refer to traditional software developers or any other business practitioner, such as actuaries or data scientist, who will take up the role of a builder.

High performing teams and organisations deliver a TTU of days rather than the weeks and months that is often the case. Similar to the other four metrics, this translates in the ability to operate at an order of magnitude difference in experience than their peers.

TTU for sustaining innovation

In his groundbreaking research on motivation, Victor Vroom of the Yale School of Management found that an individual's Expectancy, or anticipation of the connection between effort and impact, was the key foundation for long term motivation.

We've all experienced the honeymoon moment of joining a new team, project or organisation where we're full of ideas, excitement and a desire to contribute. When that energy can quickly translate into the ability to make an impact we are far more likely to sustain our sense of motivation and our hunger to innovate. When that energy hits a proverbial brick wall of bureaucracy, context overloading and panicked helplessness, the small glowing ember of innovation quickly dies.

The three biggest impediments to TTU are:

1. The how do I structure this problem: I see something that needs fixing, but how do I start making progress in formulating a solution.
2. The barriers to entry problem: When 80% of the effort is expended on solving barriers to innovation such as environmental, permissions and boilerplate code, it's very difficult to expend meaningful mental energy on formulating an innovative solution.
3. The how do I productionize and serve my solution problem: It's one thing to write code that solves a problem. It's an entirely more daunting problem to try think about how to get that solution into the hands of people that need it.

Decreasing TTU requires intentionality in improving developer experience and ruthlessly eliminating the three former mentioned impediments.

At the core of Alis Exchange, our attempt at addressing this has been the development and adoption of the Alis Build platform. The tooling, templating and simple Define, Implement and Consume framework eliminates the three frictions and enable builders to digitally innovate faster than ever before.

What is the effect of the platform on TTU? We will let our clients share that.

TTU < 5 days

A newcomer at one of our clients went from a boxed laptop and zero cloud experience to a productionized process solution in less than 5 days. This short testimonial from a new builder, Daniel, captures the excitement that comes with immediately being able to innovate and make an impact.

Read more about the case study here.

Build time: 18 months build time by 7  finance & machine learning practitioners.

The Problem

Can a machine learning enabled investment strategy beat a passive fund? Can a Machine + Human beat a traditional analysis? 🤔

This was the task that Rezco Asset Management encountered around 2015.  What initially started around a discussion on how to improve their investment idea screening tools, turned into a multi-year journey in which one had to master the intersection of machine learning, software engineering, finance and human behaviour.  Read more on our journey to scale in this memo.

We learned through experience that the engineering challenge of developing and running cutting edge machine learning models at scale was only half the problem. And in many ways it is the easier half to solve leveraging well developed machine learning cloud based IaaS solutions offered by the likes of Google Cloud, AWS and Azure.

The hardest part of the problem was learning how to frictionlessly incorporate AI research cycles and predictive signals into the workflow of Portfolio Managers (PMs). The hardest part of the problem was not figuring out how to create value through machine learning, but figuring out how to help the consumers of this technology realise the value being created in the areas most important to them.

The Solution

Meet the stakeholders where they are.

The core guiding philosophy that helped us solve this problem is that we needed to meet the stakeholders where they are. Context switching destroys focus and value realisation.  Here are a few simplified examples:

• Sending a set of stock predictions in a spreadsheet to the investment team and then asking for feedback does not work.  This is not meeting the investment team where they are, instead its essentially asking the PM to 'step away from their current context'.  PMs need to be able to make decisions in a distraction free environment which enables and maintains focus.
• Creating a "Swivel-Chair-Interface" by requiring a PM to log into some portal or pull up some email to access stock predictions.  There are already a large number of investment tools, adding another 'terminal' just to access predictions does not make sense.

Swivel chair is a slang term for a common interface work-around that involves manually entering data into one system and then entering the same data into another system.

Instead, what if the machine learning team is able to deliver predictions and well crafted model properties (features, model hypothesis, benchmark etc.) directly to into the Portfolio Manager's context?

Alis.Build enabled this through a set of consistent, well organised, well defined and highly performant APIs.  Every interaction between the stakeholders are performed through this single API layer. Practically, these 7 individuals implemented more than 200 micro services, for example services that:

• manage the integration with Morningstar Direct to generate marketing material and analytics
• building a highly scalable "Alpha Console" integrating machine learning driven predictive signals with traditional analyst research.
• manage predictions (GetPrediction, ListPredictions, UpdatePrediction, etc.)
• manage the data pipelines from Bloomberg
• capture any investment research memos and notes and propagate these up at a stock level next to prediction insights.
• etc. (more than 200 of these 😎)

The Result

Rezco Asset Management is generating outperformance on a pure Machine Learning investment strategy, after all trading costs.

This tight integration between the machine learning team and PMs creates a closed feedback loop between the machine learning and finance domains.  Better models -> more engaged PMs -> better human in the loop feedback -> Better models

In the following video, Simon Sylvester (CEO) shares how machine learning conceptually fits into their investment management world:

Watch Part 2, where you can see a practical example of how ML delivers real value to their clients

How Alis.Build made it possible

Best-in-class, resource-driven API development and cloud infrastructure management made easy for non-professional software developers with:

1. Out of the box libraries, Excel add-ins and documentation;
2. Support from the Alis Build team on thorny technical issues; and
3. Access to learning and development opportunities for the team.

The Problem

Portfolio managers live and breathe alpha. It is what gets them up in the early hours of the morning and working past midnight. It is their driving motivation, the competitive fire that burns brighter with every opportunity and every challenge.

Compliance teams, on the other hand, are primarily concerned with ensuring the portfolio in question does not fall afoul of the relevant regulation. This is no easy task and takes countless hours spent parsing obscure legislation, compiling intricate instrument master lists and formulating different tests and scenarios against which a portfolio must pass.

Typically, compliance systems and processes exist largely independent of the investment process. This creates two detractors of investment performance.

Firstly, compliance feedback on a portfolio is only delivered post trade. This leaves portfolio managers feeling like feedback is a day late and a dollar short and can cost performance through having to unwind positions.

Secondly, in the case that pre-trade feedback is provided, it is often via a separate system or set of tooling. This switching cost detracts from performance by destroying flow and focus.

The Solution

The Rezco Fund Services compliance engine is built on Alis Exchange and offers an API first implementation that:

1. Integrates into Portfolio Managers pre-trade workflow without incurring any switching costs;
2. Provides real-time compliance feedback, which does not encumber the portfolio management process; and
3. Is easy to configure and maintain with a full audit trail of every compliance check.

How does an effective, flexible, fast, workflow agnostic, API first compliance technology add alpha to the investment process? Watch the video below to find out more.

The Results

• Faster decision making.
• Reduce cost of unwinding noncompliant positions.
• Eliminate noise.
• Add compliance alpha to your process.

How Alis Build made it possible

Best-in-class, resource-driven API development and cloud infrastructure management made easy for non-professional software developers with:

1. Out of the box libraries, Excel add-ins and documentation;
2. Support from the Alis Build team on thorny technical issues; and
3. Access to learning and development opportunities for the team.

Getting data from Bloomberg Data Licence

Bloomberg Data License is the Rolls-Royce of market data providers in terms of coverage, quality and depth. It also provides a useful case study of the many hoops, contortions and escape room like puzzles that consumers of financial data are expected to solve - even from the very best of these types of systems.

The request protocol involves a cryptic request message in the form of a .txt file, that is uploaded to an SSH File Transfer Protocol (SFTP) server via a whitelisted IP address, an undisclosed waiting period and, finally, a .txt response file containing your data in yet another a cryptic format. If you find that sounds slightly complicated, not only are you right, but you’re not yet enlightened to the hidden complexities of this task. Let’s break it up into steps… (to skip the tedious explanation of the status-quo and see our solution, skip ahead to Data Engineering as a Service)

Step 1: Construct the request file

To start off with, you are required to manually construct a text file in a very particular format, indicating which Instruments and Fields you might want to get data for. For our example, we would like to request price and market cap data for Apple, Microsoft and Tesla from 1 Jan 2000 up to today (5 October 2022).

The request file, as seen in the image below is a pure text file with very strict formatting rules, including the use of a START-OF-FILE, START-OF-FIELDS and START-OF-DATA lines, nudging you to start filling in text, with the accompanying END statements, signifying you can stop now.

If a single one of these lines are out of place, or a single typo is present in the request, the entire request becomes void (but this would only become apparent in Step 4). With no linting implemented or documentation to guide you, you are left with a trial-and-error approach until the glorious day when you guessed the correct sequence of numbers and letters.

Step 2: Drop the request file on an SFTP server

The next step in the process involves making a Secure Shell (SSH) connection to Bloomberg’s remote server, in order to transfer your request file. Moreover, you can’t just do this from any old computer. Bloomberg requires any requests to be made using an IP address that they have officially white-listed. So this either implies that you either need to spin up a Virtual Machine (VM) or walk to your company’s dedicated “Bloomberg request machine”.

Generally, open-source FTP applications such as FileZilla may be used for dropping the request files. After searching for the email where your colleague sent you the SFTP credentials, an SSH connection is established and you may now manually upload your request file.

Step 3: Wait

That’s all you can do at this point. Go make yourself a coffee. Go reply to that email you’ve been ignoring. Wait for an undisclosed amount of time while Bloomberg processes your request. Every now and then, you can run back to your dedicated Bloomberg computer, SSH back onto the SFTP, and check if your data is there yet.

Step 4: Decipher the data on the reply file

The response came from Bloomberg! Success! Right?

Well, as mentioned in Step 1, your request file might have some typos or missing information, which will only become apparent by seeing Bloomberg’s response, illuminating your shortcomings. So in the trial-and-error cycle, this would be where you get the (albeit quite cryptic) error.

Let’s take the optimistic view and assume that your first attempt resulted in a positive response from Bloomberg. Surely now we’ve come to the end of our journey? Well, not quite yet. Next, you have to deal with yet another text file and figure out how this response data may be rendered useful. The infamous START-OF-FILE, START-OF-FIELDS and START-OF-DATA make their come-back, with their END counterparts indicating to us that we have come to the end of the response.

Characters such as | are used to separate the effective date and its corresponding historical value. Once you’ve deciphered the reply file, you can spend your time parsing the file, converting it to the relevant data types, and copying these into your data warehouse.

Step 5: Go home

You’ve done enough work for today. That was exhausting.

Introducing Data Engineering as a Service (DEaaS)

At Alis Exchange, we often have moments where we think: “there has to be an easier way to do this”. This is often followed by the dreaming phase, which is characterised by the phrase “imagine a world where {insert the perfect solution}”. And finally, we immerse ourselves in designing the solution, aiming wholeheartedly at making our client’s lives as easy as possible.

Imagine a world where:

• Onboarding a data source took minutes not months
• Data synchronisation never failed
• Data sources were pre-mapped to a common identifier
• Data could be consumed safely by developers in their programming language of choice without leaving their IDE

Our Data Engineering as a Service offering (DEaaS), Alis DE, turns that aspiration into a realisable reality.

A core pillar of the DE philosophy is the idea of a proxy on existing methods of data providers. We manage all the complexities of the data providers, and wrap it with a simple-to-use interface, significantly alleviating the effort required to get data into your system.

Let’s put the words into action by showing you the process of making the exact same request as above using DE… (The examples shown are in Go, although all methods are implemented and available in multiple supported languages.)

Step 1: Instantiate your Bloomberg client

Once you’re signed up as a DE client, full access to the Bloomberg methods is as simple as establishing a DE Bloomberg client in your favourite programming language. This is a once-off process, which requires minimal effort. Read more about how to make your first request to an Alis Exchange service in your favourite language.

Step 2: Make the request

Once the client is established, use the automatically generated protobufs to guide the process of constructing your request, and interpreting your response. Each request field is well defined, with documentation providing more context on how to populate the field.

DE is a cloud-native service, which ensures a scalable, readily accessible and secure service. The request will be reliably handled, without a single concern over using the correct, whitelisted computer (this is all managed in the background by DE).

// Instruments for which to request historical data
	instruments := []*pb.GetHistoryRequest_Instrument{
		{
			Id: "BBG000B9XRY4",
		},
		{
			Id: "BBG000BPH459",
		},
		{
			Id: "BBG000N9MNX3",
		},
	}
// Historical data fields requested for all instruments
fields := []*pb.GetHistoryRequest_Field{
	{
		FieldMnemonic: "PX_LAST",
		Type:          pb.FieldType_BB_DECIMAL,
	},
	{
		FieldMnemonic: "CUR_MKT_CAP",
		Type:          pb.FieldType_BB_DECIMAL,
	},
	{
		FieldMnemonic: "PX_HIGH",
		Type:          pb.FieldType_BB_DECIMAL,
	},
}

// Construct the GetHistory request message
req := pb.GetHistoryRequest{
	Headers: &pb.GetHistoryRequest_Headers{
		Firm:        "dl2345",
		ProgramFlag: pb.ProgramFlag_ADHOC,
		StartDate: &date.Date{
			Year:  2022,
			Month: 1,
			Day:   1,
		},
		EndDate: &date.Date{
			Year:  2022,
			Month: 10,
			Day:   5,
		},
	},
	Fields:            fields,
	Instruments:       instruments,
	FtpUploadRequired: true,
}

// Make the GetHistory request
res, err := BloombergClient.GetHistory(context.Background(), &req)
if err != nil {
	// TODO: Handle error
}

Step 3: Make your code do the waiting for you

Now that your request has been made, we wait for the response from Bloomberg. This time, however, DE will poll the SFTP server for you, and parse your response as soon as it is available. No checking on the FTP, and certainly no manual interpreting of the returned file.

// Wait for long-running operation to complete 
// (i.e. wait for response from Bloomberg)
//
// DE manages the polling of the SFTP, mapping the returned file 
// to the original request, and parsing the response file into a 
// well-defined response type.
resOp, err := wait(context.Background(), res)
if err != nil {
    // TODO: Handle error
}

// Marshal the long-running operation into the expected 
// response format
data := &pb.GetHistoryResponse{}
err = anypb.UnmarshalTo(resOp.GetResponse(), data,
proto.UnmarshalOptions{})
if err != nil {
    // TODO: Handle error
}

// Iterate through instruments returned by the GetHistory method
for _, instrument := range data.GetData().GetInstruments() {
    // Iterate through each record returned by the GetHistory method
    fmt.Printf("Instrument %s: Field: %s # Historical records: %v", 
        instrument.GetId(), instrument.GetFieldMnemonic(),
        len(instrument.GetHistoricalRecords())),
}

You are left with a clean response, with well-defined fields, type-safe values, and all within the confines of your IDE.

Conclusion

It is time to help along the evolution of our investment processes. Data can no longer be accessed by constructing obscure request files, performing manual processes, and interpreting unclear response files. Experience having your data to be brought to you, consumed in your language and seamlessly incorporated into your pipelines. Experience having the power of DE as a service.

After all that work, many APIs fail at the last hurdle - not realizing their value creation potential because their developers fail to consider the user experience and accessibility considerations that ultimately drive adoption.

Put simply, the easier it is for services to be used, the more they will be.

The ingredients to a great API user experience

1. Documentation

Documentation is the only window a user has into your API - it has to be done right. But, at the same time, documentation is boring to maintain and takes up developer time that could be spent on building new features and offering more value to end users.

How do we ensure our APIs have robust documentation without falling in to the deep, dark abyss of maintenance?

2. Stability and consistency

APIs should not change dramatically or quickly and should still support users longgg after they've been deprecated. API URLs and client libraries should include versions in the URL or package, and any breaking change to the API should result in a bump of the major version. Backwards compatibility is important - you can't just remove things from and change your API willy-nilly or you'll break your clients' code. Big no no.

So how do we seamlessly integrate mechanisms for implementing versioning and backwards compatibility checks to ensure things like this don't happen?

3. Meeting your user where they are

The user-experience when integrating with your API should be an enjoyable one. This means client libraries in your users' favourite language that feel natural to adopt. It doesn't mean spending hours crafting custom wrappers on REST endpoints to make these libraries available.

How do we support diverse options for accessing APIs without spending time manually writing client libraries which need to be edited every time there's a change to the API?

Enter Alis Build

The Alis Build platform makes easy to use protos to define  APIs and supercharge the user experience.

All of our APIs are defined using Protocol Buffers, which form the basis for everything that is created on Alis Exchange. After creating a proto, defining the API definition and releasing this definition, it enables us to automate many of the best practices that lie at the heart of creating great APIs.

1. Documentation

The core principles for documentation on Alis Exchange are:

1. Documentation should be generated and not maintained.
2. The process for creating documentation needs to live as close to the developer experience as possible.

Alis takes the process for creating and maintaining docs and seamlessly integrates it into the developer experience. Reference documentation is generated directly from the API definitions in your protos and custom documentation lives next to the protos. All you need to do is run an alis docs release and allow us to handle the rest. A couple files from the developer results in a fully generated docs website.

Check out docs.de.alis.services to see for yourself!

2.  Stability and consistency

Alis has backwards compatibility checks and versioning baked into our process. When running alis proto release, breaking changes are immediately detected which prevent broken client libraries and auto-generated REST APIs from making their way into production.

$ alis proto release xmpl.br.resources-books-v1
ERROR   rpc error: code = FailedPrecondition desc = protocol_buffer (protocolBuffers/xmpl.br.resources.books.v1) is not backwards compatible:
           CONFLICT: "ListBooksResponse" field: "next_page_token" has been removed, but is not reserved [books.proto]

API versioning is also enforced at the design level, as all newly created APIs must specify a version. This version is then carried through to the client packages and API URLs.

3. Meeting your user where they are

When running alis proto release client packages are immediately generated in Go, Python and Javascript, with support for many additional languages coming soon. REST APIs for traditional JSON support are also auto-generated.

Gone are the days of creating the impression that you support a range of languages when in fact you are simply making a REST call from the language. Also, no need to go through the efforts of updates when you add features to your API. You simply focus on your protos and ... I feel like I'm repeating myself now.

Ready to build your own APIs?

Want to start using next-generation tooling that will eliminate toil from your development process, turn your every-day developer into a quality software engineer and take your APIs from good to great? Reach out to us.

API is one such term. As a very simple example, an API will take in a request from a user, process the information required to answer this request and return a response back to the same user.

How is everyone an API then?

Everyday we take in requests from our colleagues, service providers and clients. Using our experience and knowledge we formulate an answer using our memories, spreadsheets, disparate pieces of software and possibly even pieces of paper. We return our answers using our voices, emails and chat messages, trusting that the requester understands what we have returned and that is it in a format that they can use.

Similar to a poorly designed API, at times our responses can come too late to be useful as we were working on another task. Or, we forgot a small important aspect that needed to be considered. In addition, how we deliver the answer and in what format can change from day to day.

Business logic to API

What would happen if we could turn our business logic into an API?
Core business functions typically  become repeatable, accessible and independent of a person or group of people. This means that effort does not have to scale linearly with business growth.

Embarking on this journey will cause one to think deeply about your process and will illuminate hidden assumptions and trade offs being made by you and your team. An additional benefit is that you will be able to identify points of failure and risk in your process.

Using Alis Build, it becomes simple to encode your business logic into APIs that can be made accessible to colleagues, service providers and clients. Grow your business without growing the often unconsidered burden of effective communication.

These silos can not only have a negative impact on the quality of solutions, but also have damaging effects on organisational culture.

But need this status quo persist?

Business requirements kill innovation

The following dance will feel all too familiar for far too many in the world of enterprise and technology:

A business team has a problem. There is some process or piece of operations that needs to be made more effective or efficient. A nice web application to replace and partially automate the incumbent Excel laden process would be nice. And so the business team does what business teams do - they write a set of business requirements. Though well meaning, these business requirements are usually flawed - they're wrong and premature.

Firstly, business teams usually do not have the necessary technical exposure to know what is possible, their solution space is limited. Secondly, little opportunity is left for experimenting and iterating on a solution in an agile manner, where both the technical and business domain knowledge areas are at hand. This will usually lead to the recreation of a spreadsheet on the internet which, in most cases, is just a worse spreadsheet.

The technical team, having received a list of features to implement rather than problems to collaboratively solve will look unquestioningly at said business requirements and then simultaneously scoff at their naivety and cringe at their impracticality. Then it's off to the salt mines to hack together just enough javascript to pass muster at the next sprint review.

Act 2 starts when the business team then receives exactly what was asked for, but observe that they are still failing to realize the efficiency and effectiveness benefits that were the original objective. The fix - more business requirements in greater detail, more scoffs and the dance continues.

The outcome? In the worst case scenario, the development team often needs to go back multiple times and rework the solution in order to meet objectives. In the best case scenario, the solution will “work”, but the organisation gains no competitive advantage since no opportunity for innovation at the intersection of the problem context and technologies/tools was created.

At Alis Exchange we hold the strong belief that in order to innovate towards the best possible solution for a problem, you need to have a good grasp of both the technologies or tools that are available, as well as a solid understanding of the problem context. A separation between these knowledge domains will ultimately lead to a subpar solution.

Damage to organisational culture

Divisions between the technical and business domains creates friction between teams and often leads to an atmosphere of outright hostility between IT and Business.

Technical teams often feel that the business teams do not have an appreciation for the work they do. Software engineering is a demanding craft and elegant, robust solutions take time and effort.

It is easy for an MBA to wave their hands and softly mutter "software software software" but those that live in the real world have to deal with scalability, extensibility, maintainability, security and a myriad of other important implementation considerations. The lack of appreciation for technical skills coupled with unrealistic expectations leads to a lack of motivation and an incentive to fight against being asked to do anything. IT teams invented soft quitting long before TikTok popularised the idea.

Business teams, on the other hand, often misinterpret technical challenges as IT "getting in the way". The sentiment is that the techies are just having fun whispering to computers while the grown ups in our corner are shouldering the burden of figuring out how to serve our customers and grow the business.

As an illustration of how deep-set this holy war goes ask yourself, "which of the above two arguments did I find myself most agreeing with?" And if the answer is either then you're part of the problem.

How do we avoid this?

Ultimately, you want to break down barriers to these knowledge areas. There are many solutions out there that are democratising the understanding and use of technologies, and there are also many ways in which organisations can give their teams more insight into the context of the problems that the business is aiming to solve.

Breaking down the barrier to building software

Given today’s accessibility of information, anyone can give themselves an education on the basic technology concepts like what an API is. It might be unrealistic to become an expert, but you should be able to have a meaningful conversation with a developer expert and understand what tools are used for which purposes. Not understanding the basics of what an API is today, is like not being able to use Microsoft Office Word 10 years ago.

Breaking down the barrier to the problem context

All individuals, including developers, working on solving a given problem, must feel the pain of the customer. Organisations should work to make customer feedback and interviews accessible to all team members. Nothing beats observing a customer in the problem context first hand! Nothing makes you understand customer paint points more than eating your own dog-food by becoming a user yourself.

Should we even have separate expert teams today?

One could argue that information or expertise on how to build software is a lot more accessible today. You do not necessarily need to be an expert to start building your first piece of software. It is however much harder to acquire a deep understanding of the problems that are experienced in a certain domain. This requires first hand experience and time.

Platforms like Alis Build also make building software a lot more accessible. Ultimately providing you with a control plane that makes it easy to configure the components you need to build robust, scalable and secure software solutions.

Organisations can afford to invest less resources into building up silos of technical expertise on how to build software and should now invest elsewhere to differentiate the quality of their solutions. What will differentiate them today is how well they can innovate at the interaction of the deep understanding of their problem domain and available tools and technologies.

Can this really be done effectively if teams operate in silos? Perhaps we have arrived at a new future where specialised silos are no longer needed and all individuals contribute to the digital economy to help solve problems?

Think this is still far away? I leave you with this example. Twenty years ago, asset managers differentiated themselves by adopting Excel as a tool. And purely having a team of experts that knew how to use Excel was a differentiator. Later on, everyone knew how to use Excel, but it was in how well you translated your problem into an Excel model that differentiated you. Needless to say, no matter how well you use Excel today, it does not really give you much advantage.

Today we again find ourselves in a place where it is not really special to have a team of developer experts that know how to build software. So what will differentiate us?

Why is it so hard to name things? Or more precisely, why is it so hard to pick a set of symbols that reliably communicate the intended meaning to the interpreting audience? And why is it particularly difficult to name things within information systems?

Let’s unpack the three tensions of naming that excellence that underly these difficulties:

Bridging the Tech / Business divide

Ill considered names exacerbate divides. Usually the culprit is a technologist lazily exporting their database schema.

Excellence requires building definitions around deep understanding of the words used by domain practitioners and the meaning attached to these words.

At the same time, definitions must be usable. This means implementing a definition in such a way that it can be efficiently modeled and typed.

Balancing stability with extensibility

Well developed definitions are stable but extensible. Too much change creates chaos. Rigidity means that the world will pass you by.

Goldilocks specificity

Too general, and what’s the point. Too specific and what’s the point. The point of a well defined definition layer is the “not too hot”, “not too cold” level of compromise of standardization enabling the specific needs of individual use cases.

Let’s take an example: defining an “Instrument” in the asset and wealth management industry

First, what is it that we are actually trying to define here. Going from, "we all implicitly know what this is" to "this is what we are actually trying to define here" is difficult.

// An Instrument is a financial asset or series 
// such as an equity, bond, commodity, currency pair or index. 
message Instrument {
	// TODO: let's define an instrument
}

Next, let's decide how this thing will be named. Immediately things get tricky. The name itself carries information. Across Alis Exchange we strongly encourage the use of Resource Oriented Design naming conventions (read more here) to help users more easily intuit the collection or sub-collection a resource belongs to and anticipate how to use and work with these names.

In this case, any user can rely on the fact that the ID portion of an instrument name is either a FIGI from the OpenFIGI database or a custom created FIGI that obeys the same naming conventions (read more here) and reason accordingly.

// An Instrument is a financial asset or series 
// such as an equity, bond, commodity, currency pair or index. 
message Instrument {
	// The instrument name is the unique identifier across organisations.
    // Format: instruments/{instrument}. 
	// {instrument} must be a valid Bloomberg or custom FIGI,
    // as defined by www.openfigi.com..
    string name = 1;
}

Now we start to add the fields that define an Instrument object. The security description and display_name intentionally deviate from the OpenFIGI specification. We thoughtfully decided to deviate from the definition after weighing the needs of users of the system against the value of adhering to a global specification.

For some fields, it is also helpful to enumerate possible values. For example, we want to restrict Market Sector to a set of choices that data producers and consumers can more easily reason about. Safety and usability often go hand in hand.

// Instrument is a resource in the IN domain.
message Instrument {
    // ...
    // Figi Type. Either a global or custom FIGI, else set to unspecified
    FigiType figi_type = 2;
    // Human readable display name
    string display_name = 3;
    // Security Description
    string description = 4;
    // Market Sector
    // The asset class/category of the particular instrument
    // Format: EQUITY, MONEY_MARKET, etc.
    MarketSector market_sector = 5;
    // The instrument market sector
    enum MarketSector {
        // Market Sector not specified
        MARKET_SECTOR_UNSPECIFIED = 0;
        // Commodities
        COMMODITY = 1;
        // Corporate Bonds
        CORPORATE = 2;
        // Currency
        CURRENCY = 3;
     //...
}

It is important to know where to draw the boundary of a definition. For example, in an early version of the API definition we added fields that would allow users to store instrument data relevant to their compliance process. We later realized that this data would be better managed in a separate service and therefore decided to safely deprecate the field.

// Instrument is a resource in the IN domain.
message Instrument {
    // ...
    // Compliance attributes
    // A set of attributes specific to the compliance domain
    Compliance compliance = 13 [deprecated = true];
    // All compliance related attributes.
    message Compliance {
        // UCITS attributes.
        Ucits asisa = 1;
    // ...
}

And finally, a well designed API bridges the Tech / Business divide by well documenting domain relevant information within the API. For example, the Instrument definition has a set of fields that enable the user to store associations to other external identifiers used by market data providers (such as SEDOL, CUSIP, RIC and PermId). These fields are well documented with the information a user needs to understand the field and a reference to sources of further information where applicable.

// Instrument is a resource in the IN domain.
message Instrument {
    // ...
    // External identifiers
    ExternalIDs external_ids = 8;
    // External identifiers
    message ExternalIDs {
        // figi as defined by www.openfigi.com.
        // If no figi is provided, a custom figi will be created using the remaining attributes.
        string figi = 1;
        // Composite Financial Instrument Global Identifier - The Composite Financial Instrument Global Identifier
        // (FIGI) enables users to link multiple FIGIs at the trading venue level within the same country or market
        // in order to obtain an aggregated view for an instrument within that country or market.
        string composite_figi = 2;
        // Share Class Financial Instrument Global Identifier - A Share Class level Financial Instrument Global Identifier
        // is assigned to an instrument that is traded in more than one country. This enables users to link multiple
        // Composite FIGIs for the same instrument in order to obtain an aggregated view for that instrument across all
        // countries (globally).
        string share_class_figi = 3;
        // ISIN
        // An International Securities Identification Number (ISIN) is a code that uniquely identifies a specific
        // securities issue.
        string isin = 4;
        // SEDOL (Stock Exchange Daily Official List)
        // a list of security identifiers used in the United Kingdom and Ireland for clearing purposes.
        // The numbers are assigned by the London Stock Exchange, on request by the security issuer.
        string sedol = 5;
        // CUSIP
        // a nine-digit numeric (e.g. 037833100 for Apple) or nine-character alphanumeric (e.g. 38259P508 for Google)
        // code that identifies a North American financial security for the purposes of facilitating clearing and
        // settlement of trades.
        // https://en.wikipedia.org/wiki/CUSIP
        string cusip = 6;
        // RIC (Reuters instrument code)
        // is a ticker-like code used by Refinitiv to identify financial instruments and indices.
        // https://en.wikipedia.org/wiki/Reuters_Instrument_Code
        string ric = 7;
        // Primary RIC (Reuters instrument code)
        // is a ticker-like code used by Refinitiv to identify financial instruments and indices.
        // https://en.wikipedia.org/wiki/Reuters_Instrument_Code
        string primary_ric = 8;
        // Bloomberg Ticker
        // a string of characters or numbers to identify a company or entity uniquely in Bloomberg.
        string bloomberg_ticker = 9;
        // MSCI Security Code is a unique identifier assigned to each security by MSCI.
        string msci_sec_code = 10;
        // A Central Index Key or CIK number is a number given to an individual, company, or foreign government by
        // the United States Securities and Exchange Commission. The number is used to identify its filings in
        // several online databases, including EDGAR.
        string cik = 11;
        // InvestOne Id
        // InvestOne is an investment accounting solution for institutional asset managers.
        string invest_one = 12;
        // Morningstar SecId
        string morningstar_sec_id = 13;
        // Morningstar CompanyId
        string morningstar_company_id = 14;
        // MCH Id
        // MCH (Multi-Currency Horizon) is an investment accounting solution for institutional asset managers.
        string mch = 15;
        // Multifonds Id
        // Multifonds is an investment accounting solution for institutional asset managers.
        string multifonds = 16;
        // Fundamental Id
        // Fundamental is an investment accounting solution for institutional asset managers.
        string fundamental = 17;
        // Morgan Stanley Id
        // Morgan Stanley is an investment trading solution provider for institutional asset managers.
        string morgan_stanley = 18;
        // JSE Alpha Code
        // The ticker used by the Johannesburg Stock Exchange
        string jse_alpha_code = 19;
        // Refinitiv PermID
        // PermIDs are open, permanent and universal identifiers where underlying attributes capture the
        // context of the identity they each represent.
        // https://permid.org/
        string perm_id = 20;
        // Datastream Symbol
        string datastream_symbol = 21;
        // IEX Identifier
        string iex_id = 22;
    }
    // ....
}