It’s not hyperbole to say that ChatGPT is revolutionary. It’s a significant leap over the often frustrating experience of using Siri or Alexa.
Like Siri and Alexa, ChatGPT is doesn’t always get everything right. It can state nonesense confidently as if it were fact. However, whether it’s searching the web, reading a newspaper article, or even an academic text, I’m quite used to having to have my bullshit detector switched on and comparing several sources to verify things. In that respect, ChatGPT seems as good as any other information source. (It got almost everything about my areas of expertise — the Scots language, software development and music — correct.)
Since its launch late last year I’ve been thinking about whether ChatGPT might free white-collar workers (or middle-class professionals, as you might call them here in the UK) from everyday office drudgery. We’ve been getting there over time: my first office job 20 years ago, where I transferred information from scanned paper copies of life insurance applications into text fields in a desktop software application by typing a letter at a time, surely no longer exists.
Could ChatGPT be a revolutionary, as opposed to evolutionary, step in freeing us from monotonous work (not before UBI though, please; I have a mortgage to pay)? I have questions about what AI tools might mean for content producers, office workers and even the future of society. I decided to put some of them to ChatGPT directly to see what it had to say. It was an informative chat. A transcript is below. The original text can be found on the OpenAI website.
Will ChatGPT ever list its sources so users can verify the information and do further reading? permalink
ChatGPT is trained on a diverse range of internet text, which means it can generate responses based on the information it has seen during training. However, it does not store or have a way to retrieve the specific source of the information it provides. While it is not possible for ChatGPT to list its sources for users to verify information or do further reading, it is possible to develop a system that would keep track of the sources used to train a certain model version, so that users could identify potential sources of the information generated by the model. This can be done by keeping a record of the data used to train the model, but this would require significant computational resources and would make the model less efficient.
It‘s important to note that GPT-3 and other models like it should be considered as an aid and not a replacement for human knowledge and critical thinking. They should not be used as a sole source of information and their outputs should always be critically evaluated.
Will providers of the information used by ChatGPT be compensated in some way? For example, will a[n] academic receive any royalties if several of their papers contributed to the model? permalink
It is unlikely that individual authors or sources of the text used to train ChatGPT or other language models will be compensated directly. The vast majority of the text used to train these models is taken from publicly available sources on the internet, such as websites and social media, which are generally not covered by copyright. Additionally, it would be extremely difficult to track down and compensate all of the individual authors and sources of the text used to train a model, especially given the sheer volume of text that is used.
In the case of academic papers, it‘s possible that the authors of these papers would not receive any royalties or compensation from the use of their papers in the training data set. Many academic papers are made available online for free and the authors have likely signed over the copyright to the publisher.
It‘s worth noting that OpenAI, the company behind ChatGPT, has a policy of only using text that is publicly available and not covered by copyright. Additionally, OpenAI has also committed to using a portion of its revenue to fund research and development in AI safety and open AI.
It‘s important to note that the use of text from any sources, whether they are publicly available or not, should always be done in compliance with the terms of use and copyright laws.
Could ChatGPT ingest highly technical sources that cover a particular domain, for example aerospace engineering, to create a custom model for use within a particular industry? permalink
Yes, it is possible to fine-tune ChatGPT or other language models on a specific domain or industry, such as aerospace engineering. This process is called “domain adaptation” or “domain transfer.” By fine-tuning the model on a specific dataset of domain-specific text, the model can learn the specific terminology, concepts and writing style of that domain, which allows it to generate more accurate and relevant responses when working within that domain.
Fine-tuning can be done by using a dataset of technical documents, articles or papers that cover the specific domain such as aerospace engineering. The process involves training the model on this specific dataset, which allows it to adapt to the specific characteristics of the domain.
However, it‘s worth noting that fine-tuning a model on a specific domain requires a large amount of high-quality, domain-specific data and computational resources. Additionally, the fine-tuned model will perform best on the specific task and domain it was trained on and may not generalize well to other tasks or domains.
It‘s also important to keep in mind that a fine-tuned model may not perform as well as an expert in the field, and the outputs should be always evaluated critically.
Can you give a ballpark figure as to how much data would be required? For example, would 250,000 words be enough? permalink
The amount of data required to fine-tune a language model on a specific domain depends on various factors such as the complexity of the domain, the quality of the data, and the size of the pre-trained model.
In general, the more data and the more diverse the data, the better the fine-tuned model will perform. However, as a rough estimate, around 250,000 words may be enough to fine-tune a model on a specific domain, but it‘s also possible that more or less data may be needed depending on the specific situation.
For example, for a specific task or domain like aerospace engineering, 250,000 words of high-quality, domain-specific text such as academic papers, technical reports and journal articles would likely be sufficient to fine-tune a large pre-trained language model like GPT-3. However, for a more complex and broader domain like medicine, it may require more data to achieve similar results.
It‘s worth noting that in practice, obtaining a large amount of high-quality, domain-specific data can be a challenging task and requires a considerable effort. Additionally, it‘s also important to keep in mind that fine-tuning a model on a specific domain requires computational resources and time, and the performance may vary depending on the specific task and data.
Can ChatGPT understand diagrams as well as standard text/prose? permalink
ChatGPT is a language model, which means it is trained to understand and generate text. It is not specifically designed to understand diagrams or images, however, it can be fine-tuned on a dataset that includes images and captions, allowing the model to learn some level of understanding of image-text associations.
There are other models like image captioning models, which can be used in conjunction with language models like ChatGPT to understand and generate captions for images. These models are trained to understand the content of images and generate text descriptions of them.
It‘s worth noting that current state-of-the-art models for image captioning still have limited understanding of complex images and there is still a lot of research being done in this field to improve their ability to understand and generate captions for images.
Moreover, it‘s also important to keep in mind that a model‘s understanding of images is limited to the data it has seen during training, and it may not generalize well to new images or situations.
If ChatGPT doesn‘t know the answer to a user query, will it just say so? Would it ever give out a false answer if it didn‘t know the correct answer? permalink
ChatGPT is a machine learning model, which means that it is able to generate responses based on the patterns it has learned during training. However, it is not infallible and may not know the answer to every question or provide a false answer.
If ChatGPT doesn‘t know the answer to a user‘s query, it will likely generate a response that indicates that it doesn‘t have enough information to provide an answer. For example, it may say something like “I‘m sorry, I don‘t know the answer to that question” or “I don‘t have enough information to provide an accurate answer.”
It‘s also worth noting that ChatGPT is trained on a vast amount of text data from the internet, and the quality of the information it has seen during training may vary. Therefore, it‘s possible that the model may generate a response that is incorrect, out-of-date or biased.
It is important to keep in mind that the output of a language model should be critically evaluated, and the model should not be considered as a sole source of information or knowledge.
OpenAI, the company behind ChatGPT, has an API which allows developers to access the model‘s capabilities and integrate them into their own applications, and they also provide a way to flag unsafe or unreliable content, which OpenAI team will review and take action if necessary.
How do you think ChatGPT will impact software development? permalink
ChatGPT and other large language models like it have the potential to greatly impact software development in several ways:
- Automated content generation: ChatGPT and other language models can be used to generate large amounts of high-quality text, such as documentation, product descriptions, and even code. This can greatly reduce the time and effort required for content creation and allow developers to focus on more complex tasks.
- Improved natural language processing: ChatGPT and other language models can be used to improve natural language processing (NLP) capabilities in a wide range of applications, such as chatbots, virtual assistants, and sentiment analysis. This can make it easier for developers to create NLP-enabled applications that are more accurate and user-friendly.
- Improved search: ChatGPT and other language models can be used to improve the quality of search results by better understanding the intent behind user queries. This can make it easier for developers to create search applications that are more accurate and relevant.
- Improved data analysis: ChatGPT and other language models can be used to analyze and extract insights from large amounts of text data. This can make it easier for developers to create data-driven applications that are more accurate and informative.
- Improved accessibility: ChatGPT and other language models can be used to improve accessibility for people with disabilities, such as those who are visually impaired or have difficulty reading. This can make it easier for developers to create applications that are more inclusive and accessible to a wider range of users.
Overall, ChatGPT and other large language models have the potential to greatly accelerate
