In this paper, we propose a market model which is based on reputation and reinforcement learning algorithms for buying and selling agents. Three important factors: quality, price and delivery-time are considered in the model. We take into account the fact that buying agents can have different priorities on quality, price and delivery-time of their goods and selling agents adjust their bids according to buying agents preferences. Also we have assumed that multiple selling agents may offer the same goods with different qualities, prices and delivery-times. In our model, selling agents learn to maximize their expected profits by using reinforcement learning to adjust product quality, price and delivery-time. Also each selling agent models the reputation of buying agents based on their profits for that seller and uses this reputation to consider discount for reputable buying agents. Buying agents learn to model the reputation of selling agents based on different features of goods: reputation on quality, reputation on price and reputation on delivery-time to avoid interaction with disreputable selling agents. The model has been implemented with Aglet and tested in a large-sized marketplace. The results show that selling/buying agents that model the reputation of buying/selling agents obtain more satisfaction rather than selling/buying agents who only use the reinforcement learning.
Key words: Reputation, Reinforcement Learning, Electronic Commerce Agents
(ProQuest-CSA LLC: ... denotes formulae omitted.)
With the advent of mobile and intelligent agent technology, e-commerce has been entered in a new era of its life . Also agent architecture provides a flexible environment to model the other fields of research , , . Agent- Based e-Marketplace is one of the most important results of using agent technology over e-Commerce. Electronic marketplace provides a single location for many buyers and sellers to congregate electronically and complete their own transactions. In the recent years, the extensive research is focused on designing agent-based e-Marketplaces , , , , . Moreover, there are some research on personal intelligent agents for e-commerce applications , , , , . But the most important problem that can be mentioned in these works is poor intelligence of trading agents.
In addition, reinforcement learning  has been studied for various multi-agent problems , , , . However, these efforts are not directly modeled as economic agents and market environments. There are some research on reputation and trust modeling which do not use reinforcement learning , , , , . A number of agent models for electronic market environments have been proposed. Jango  is a shopping agent that assists customers in getting product information. Given a specific product by a customer, Jango simultaneously queries multiple online merchants (from a list maintained by NetBot, Inc.) for the product availability, price, and important product features. Jango then displays the query results to the customer. Although Jango provides customers with useful information for merchant comparison, at least three shortcomings may be identified: (i) The task of analyzing the resultant information and selecting appropriate merchants is completely left for customers, (ii) The algorithm underlying its operation does not consider product quality which is of great importance for the merchant selection task, (iii) Jango is not equipped with any learning capability to help customers choose more and more appropriate merchants. Another interesting agent model is Kasbah , designed by the MIT Media Lab. Kasbah is a multi-agent electronic marketplace where selling and buying agents can negotiate with one another to find the "best possible deal" for their users. The main advantage of Kasbah is that its agents are autonomous in making decisions, thus freeing users from having to find and negotiate with buyers and sellers. …