Speeding up the RFP process using data from past bids

  • Company

    Infrastructure provider
    (global)

  • AI method

    NLP, Similarity Search

  • Industry

    Infrastructure

Most people working in services or customised systems will recognise the term RFP (or Request for Proposal). Whether it is associated with exciting projects or repetitive bid-writing processes will vary, but most will agree that making it more effective would be a welcome development.

02/ Bid-writing as a service

As the name suggests, a Request for Proposal is a document that solicits proposals from potential suppliers. Suppliers often have teams working on just RFPs to make sure they have a healthy pipeline of work lined up. One such supplier reached out to Peltarion to see if there was a way for us to make this process more efficient for them. With heaps of data from past RFPs, they thought this might be a viable starting point for training a deep learning model which could reduce the workload for the bid writers.

03/ The 'before' shot

The supplier we worked with on this particular project kept all previous RFPs in different spreadsheets. Depending on the type of requirement the RFP was asking for, the bid-writer could then scroll to find all responses relevant to that particular requirement and read through them to find the one most relevant for this particular situation. It was repetitive work that took a lot of time - usually a good starting point for automation opportunities.

04/ Making the magic happen

One of the main challenges in setting up a solution for this was that although there was a lot of data, this wasn’t actually labeled. Using a pre-trained language model, BERT, we didn’t need a labeled dataset but could instead fine-tune the model on the task of finding similar requirements using similarity search methods

Another issue was that the language used was highly specific and very different from typical English language. To interpret the acronym-heavy language in RFPs, we added these words to the vocabulary and made a custom language model for the client.

05/ Post AI-makeover

Check out the video to see the results.

06/ What else can be done

Using similarity search in this way offers a wealth of other opportunities for those who have been struggling with unlabeled datasets. Instead of using labeled data, the algorithm is told to cluster the data according to patterns it finds and use this to make decisions on how it all fits together. 

For industries that have a lot of unlabeled text data, this would be a good chance to start for exploring deep learning opportunities. See below for some examples:

  • Law - including contract generation
  • Research - for literature reviews, or even staying on top of academic papers in hotly debated topics like the public health debate around Corona (where, according to the Centre for Global Development, by July you would have had to read a new paper every three minutes to stay on top of the debate)
  • Other industries that use bid-writing processes