Connect with us


AI models are becoming better at answering questions, but they’re not perfect



Let the OSS Enterprise newsletter guide your open source journey! Sign up here.

Late last year, the Allen Institute for AI, the research institute founded by the late Microsoft cofounder Paul Allen, quietly open-sourced a large AI language model called Macaw. Unlike other language models that’ve captured the public’s attention recently (see OpenAI’s GPT-3), Macaw is fairly limited in what it can do, only answering and generating questions. But the researchers behind Macaw claim that it can outperform GPT-3 on a set of questions, despite being an order of magnitude smaller.

Answering questions might not be the most exciting application of AI. But question-answering technologies are becoming increasingly valuable in the enterprise. Rising customer call and email volumes during the pandemic spurred businesses to turn to automated chat assistants — according to Statista, the size of the chatbot market will surpass $1.25 billion by 2025. But chatbots and other conversational AI technologies remain fairly rigid, bound by the questions that they were trained on.

Today, the Allen Institute released an interactive demo for exploring Macaw as a complement to the GitHub repository containing Macaw’s code. The lab believes that the model’s performance and “practical” size — about 16 times smaller than GPT-3 — illustrates how the large language models are becoming “commoditized” into something much more broadly accessible and deployable.

Answering questions

Built on UnifiedQA, the Allen Institute’s previous attempt at a generalizable question-answering system, Macaw was fine-tuned on datasets containing thousands of yes/no questions, stories designed to test reading comprehension, explanations for questions, and school science and English exam questions. The largest version of the model — the version in the demo and that’s open-sourced — contains 11 billion parameters, significantly fewer than GPT-3’s 175 billion parameters.

Given a question, Macaw can produce an answer and an explanation. If given an answer, the model can generate a question (optionally a multiple-choice question) and an explanation. Finally, if given an explanation, Macaw can give a question and an answer.

“Macaw was built by training Google’s T5 transformer model on roughly 300,000 questions and answers, gathered from several existing datasets that the natural-language community has created over the years,” the Allen Institute’s Peter Clark and Oyvind Tafjord, who were involved in Macaw’s development, told VentureBeat via email. “The Macaw models were trained on a Google cloud TPU (v3-8). The training leverages the pretraining already done by Google in their T5 model, thus avoiding a significant expense (both cost and environmental) in building Macaw. From T5, the additional fine-tuning we did for the largest model took 30 hours of TPU time.”

Allen Institute Macaw

Above: Examples of Macaw’s capabilities.

Image Credit: Allen Institute

In machine learning, parameters are the part of the model that’s learned from historical training data. Generally speaking, in the language domain, the correlation between the number of parameters and sophistication has held up remarkably well. But Macaw punches above its weight. When tested on 300 questions created by Allen Institute researchers specifically to “break” Macaw, Macaw outperformed not only GPT-3 but the recent Jurassic-1 Jumbo model from AI21 Labs, which is even larger than GPT-3.

According to the researchers, Macaw shows some ability to reason about novel hypothetical situations, allowing it to answer questions like “How would you make a house conduct electricity?” with “Paint it with a metal paint.” The model also hints at awareness of the role of objects in different situations and appears to know what an implication is, for example answering the question “If a bird didn’t have wings, how would it be affected?” with “It would be unable to fly.”

But the model has limitations. In general, Macaw is fooled by questions with false presuppositions like “How old was Mark Zuckerberg when he founded Google?” It occasionally makes errors answering questions that require commonsense reasoning, such as “What happens if I drop a glass on a bed of feathers?” (Macaw answers “The glass shatters”). Moreover, the model generates overly brief answers; breaks down when questions are rephrased; and repeats answers to certain questions.

The researchers also note that Macaw, like other large language models, isn’t free from bias and toxicity, which it might pick up from the datasets that were used to train it. Clark added: “Macaw is being released without any usage restrictions. Being an open-ended generation model means that there are no guarantees about the output (in terms of bias, inappropriate language, etc.), so we expect its initial use to be for research purposes (e.g., to study what current models are capable of).”


Macaw might not solve the current outstanding challenges in language model design, among them bias. Plus, the model still requires decently powerful hardware to get up and running — the researchers recommend 48GB of total GPU memory. (Two of Nvidia’s 3090 GPUs, which have 24GB of memory each, cost $3,000 or more — not accounting for the other components needed to use them.) But Macaw does demonstrate that, to the Allen Institute’s point, capable language models are becoming more accessible than they used to be. GPT-3 isn’t open source, but if it was, one estimate pegs the cost of running it on a single Amazon Web Services instance at a minimum of $87,000 per year.

Allen Institute Macaw

Macaw joins other open source, multi-task models that have been released over the past several years, including EleutherAI’s GPT-Neo and BigScience’s T0. DeepMind recently showed a model with 7 billion parameters, RETRO, that it claims can beat others 25 times its size by leveraging a large database of text. Already, these models have found new applications and spawned startups. Macaw — and other question-answering systems like it — could be poised to do the same.

Source link


Seoul court rejects warrants for former Terraform Labs employees and investors over Luna collapse  • TechCrunch



A Seoul court rejected a request from prosecutors for warrants to detain eight people related to Terraform Labs, including the co-founder of Terraform Labs, Daniel Shin, early investors and former engineers.

It’s difficult to believe they would flee or destroy evidence as Shin and the seven other suspects have been cooperating with the investigation, Yonhap News said, citing the Seoul court. In addition, the suspects also need to be guaranteed their rights to defend themselves against the allegations of capital market rules, which is the core accusation of this case, according to the court, per Yonhap

The Seoul Southern District Prosecutors Office told TechCrunch that it is hard to understand that conclusion as the court knows the seriousness of the allegation and the fact that some of the suspects allegedly made money by selling Luna tokens before the collapse. And yet, the court dismissed the warrants, saying the eight people need to have rights to defend their cases against accusations.

Shin is being charged with taking illegal profits worth about $105 million by selling Luna tokens when it was near its all-time high without disclosing this move to investors. It happened before the collapse of the TerraUSD and Luna earlier this year, contravening the Capital Market Act. Prosecutors also suspect Shin used customer data from his separate fintech startup called Chai to promote Luna, violating the Electric Financial Transaction Act. The other seven people involved in Terraform were also alleged to have similar charges.

Shin has denied the claims of trading Luna at a market high and violating the customers’ data. Terraform was founded in Singapore in 2018 by Do Kwon and Shin. Shin left Terraform in March 2020 to found Chai and stepped down as CEO of Chai earlier this year.

South Korean prosecutors began the investigation after the crash of the UST-Luna token earlier this year, which wiped out $40 billion in market value. In September, South Korea issued an arrest warrant for another co-founder, Kwon, whose whereabouts are currently unknown, and requested Interpol, the international law enforcement agency, to issue a red notice for Kwon.

Terraform Labs could not be reached for comment.

Source link

Continue Reading


Amazon may lay off 20,000 employees, including managers: Report



Amazonmay lay off about 20,000 employees across divisions as the company reevaluates its pandemic-induced hiring spree, according to a media report.

A Computerworld report stated that the tech giant could lay off employees across the company, including distribution centre workers, technology staff, and corporate executives. Staff at all levels will likely be affected, it found.

Last month, the New York Times reported that Amazon plans to lay off approximately 10,000 people, and “the cuts will focus on Amazon’s devices organisation, including the voice-assistant Alexa, as well as at its retail division and in human resources”.

However, according to Computerworld, the layoffs could impact nearly double the number of employees– roughly 6% of the company’s corporate employees and about 1.3% of its global workforce of more than 1.5 million composed primarily of hourly workers.

YourStory could not independently verify the report.

Corporate staff have been told that employees will receive a 24-hour notice and severance pay, in accordance with their company contracts, the Computerworld report noted. “There is a sense of fear among employees in the company as the news has come out,” the report added, quoting a source who was informed directly about the layoff effort.

The layoffs would be the largest staff reduction in Amazon’s history.

“There is no specific department or location mentioned for the cuts; it is across the business. We were told this is as a result of over-hiring during the pandemic and the need for cost-cutting as the company’s financials have been on a declining trend,” the source told Computerworld.

After the New York Times report, Amazon Chief Executive Officer Andy Jassy shared some information about role eliminations in a note. Jassy confirmed that layoffs were occurring, though he did not specify the planned number of employees to be laid off.

“Our annual planning process extends into the new year, which means there will be more role reductions as leaders continue to make adjustments. Those decisions will be shared with impacted employees and organisations early in 2023,” Jassy wrote in the message, noting that Amazon had already communicated that layoffs would occur in the Devices and Books businesses, and would be extending a voluntary reduction offer for some employees in the People, Experience, and Technology (PXT) organisation. 

“We haven’t concluded yet exactly how many other roles will be impacted (we know that there will be reductions in our Stores and PXT organisations), but each leader will communicate to their respective teams when we have the details nailed down,” Jassy noted.

Meanwhile, the Computerworld report noted that employees on Amazon’s robotics team have been laid off.

Amazon’s muted third-quarter earnings as well as disappointing fourth-quarter projections led the company’s stock to plummet. Its third-quarter earnings were severely impacted by unpredictable consumer shopping habits and inflation. 

Amazon is likely to lay off several employees in India across divisions, according to media reports. Last month, Amazon confirmed that it will shut down its wholesale unit Amazon Distribution. This is the third business unit to be closed after the e-commerce giant announced the wrapping up of Amazon Academy and the food delivery business in India.

Globally, tech companies have announced layoffs as part of their cost-cutting efforts. In November, Meta CEO Mark Zuckerberg announced that the company had decided to reduce the size of its team by about 13%, cutting over 11,000 jobs. In the same month, Elon Musk reduced half of Twitter’s workforce or about 3,700 jobs at the social media firm.

Source link

Continue Reading


Unlock The Entrepreneurial Potential Of Your Team With Employee-Ownership



A strong team of many outperforms even the most hardworking of entrepreneurs on their own. But when hiring employees, freelancers and contractors, how do you ensure they have the same entrepreneurial skills and drive that you do as your company’s owner? Is it unrealistic to expect employees to be motivated and committed to an organisation they didn’t found?

Nicki Sprinz thinks she has cracked the code of unlocking the entrepreneurial potential of your team, and the answer lies in employee ownership. Sprinz is managing director of B-Corp certified ustwo London, a company of over 200 employees, and cofounder of Ada’s List, an 8000-strong community designed to support women working in the tech industry. ustwo has recently become employee-owned and has already seen the benefits of breaking down the distinction between owners and employees.

According to the Employee Ownership Association, this way of working can improve productivity, support more resilient regional economies and empower team members, resulting in them being far more engaged. Sprinz explained the main benefit for entrepreneurs of this model along with practical tips for managing directors and company founders to make the transition to becoming employee-owned.

Employee ownership protects the company

“Being employee-owned means existing team members, who are now partners, feel empowered as owners,” said Sprinz. She believes that this encourages everyone to put in the work to uphold a strong company culture and course-correct if they see anything awry.

Whilst this might not happen automatically, a founder can make it more likely that their team upholds the vision. Sprinz has put frameworks in place to ensure everyone has a voice. “We hold open firesides, have elected partner representatives on the board, and ensure there are regular channels of communication for all team members to be part of growing the culture and living the values,” she said.

Keeping the team on board means protecting the company. “There are no surprises about the direction we are taking with the business,” explained Sprinz. “We involve everyone in the decisions we make on our projects and ensure we are accountable, both commercially and ethically.”

Attract and retain top talent

In a competitive market, how does your company attract and retain the best talent in the world for the benefit of your clients? Employee-ownership could be the solution. Not only does it make job listings stand out, but it attracts individuals who are like-minded and think long term. They are committed to a future with whichever company they choose to join and are prepared to push themselves to make it happen.

“High quality potential recruits and employees are interested in values and purpose,” said Sprinz. “Being able to talk about employee ownership helps you stand out in a tough hiring market. We have several interview stages so a candidate can get to know us as well as we’d like to know them.”

Sprinz’ interview stages aim to weed out “cultural and value mismatches that ultimately lead to an unfulfilled team.” They ask candidates multiple questions about their values and examples of them in practice, and they encourage candidates to probe with questions about ustwo. They also “publicise the salary for all open roles and candidates have the opportunity to meet other members of the team,” she added.

Control quality

When scaling a business, ambitious entrepreneurs cannot afford to let quality slip. Growth at all costs is a false economy that ends with the business back at square one and having to work harder to undo reputational damage. “A more entrepreneurial team ensures quality stays high,” explained Sprinz. Not only do your team members care deeply about the work they do, they also know they benefit from company growth, so they are incentivised to keep raising the bar.

“If your team is invested in the long term financial success of the company, they also feel pride that their work contributes to overall success,” said Sprinz. “They respond by raising the bar on their work.” Sprinz also believes that, “Regular transparent sharing of financial results and metrics maintains dialogue on personal and company impact.”

Direct the future

An employee-owned company has options for the future. The owner might one day want to step aside or sell, and the company’s succession plan will already be in place. In the meantime, the company has hit new heights and progressed with new ideas because its foundations are solid.

Like Maslow’s Hierarchy of Needs, you cannot reach self-actualisation without warmth and shelter, and a company cannot break through ceilings with constant recruitment issues. When team members are bought into the company, they are bought into its future too, making more certain outcomes for everyone involved.

“The partner representatives on the board surface the priorities of the rest of the team and ensure the conversations of the board are directed accordingly,” explained Sprinz. “The representatives are actively part of the bigger picture and playing a huge part in shaping the company’s future.”

Unlock the entrepreneurial potential of your team by exploring employee ownership, advised Sprinz. The best people will be proud to tell their friends that they are part-owners of the place they work. They will feel valued and listened to and respond with their effort and devotion. Could employee ownership be the right step forward for you?

Continue Reading