30 Commits

Author SHA1 Message Date
lieght
22a73fc50a Moved back in threads 2026-01-23 19:16:29 +01:00
lieght
af9a316639 Merge remote-tracking branch 'origin/Development' into Development 2026-01-23 19:15:55 +01:00
lieght
8508377cb4 Parameters added to tests 2026-01-23 19:15:29 +01:00
ramollia
97276c7e80 readded threads argument 2026-01-23 19:14:46 +01:00
ramollia
11eda3c8b5 Merge remote-tracking branch 'origin/Development' into Development
# Conflicts:
#	game/src/main/java/org/toop/game/players/ai/mcts/MCTSAI3.java
#	game/src/main/java/org/toop/game/players/ai/mcts/MCTSAI4.java
2026-01-23 19:09:51 +01:00
ramollia
039c0393c8 fixed extra wait time for threads 2026-01-23 19:09:28 +01:00
lieght
319eaa33b1 name fixes 2026-01-22 13:48:15 +01:00
lieght
940b26de41 Fixes 2026-01-22 13:47:42 +01:00
lieght
2f161bcc0a Small thread count fix 2026-01-22 13:38:19 +01:00
lieght
ccdca4f0ea Added thread test 2026-01-22 12:30:42 +01:00
lieght
b5bd4adf91 AI wait fixes 2026-01-22 11:10:55 +01:00
lieght
f37c578a35 name fixes 2026-01-22 11:04:08 +01:00
lieght
eb1784550f Data collection fixes 2026-01-22 10:56:55 +01:00
lieght
e8f66a62f0 AI data now correct 2026-01-22 10:44:54 +01:00
lieght
4926bd161e m4 nu 8 threads 2026-01-22 10:29:22 +01:00
lieght
b39659d02d Back to 10ms 2026-01-22 00:00:57 +01:00
lieght
c107e8c0d1 Correct time data visualization effect on timetable lookup for data collection purposes 2026-01-21 23:52:29 +01:00
lieght
8a94aad622 Infinite game collection 2026-01-21 23:46:59 +01:00
lieght
992523b936 Better data collection for overnight run 2026-01-21 23:43:11 +01:00
ramollia
5d2fff7ae7 changed the way multithreading worked 2026-01-21 20:05:19 +01:00
ramollia
f168b974ab Merge remote-tracking branch 'origin/Development' into Development 2026-01-21 15:42:08 +01:00
ramollia
057487e4f9 readded the exploration constant 2026-01-21 15:40:38 +01:00
michiel
fb32bc6f8e saving games data to games.csv 2026-01-20 13:41:10 +01:00
ramollia
4c8bd89a35 fixed things 2026-01-20 13:18:29 +01:00
ramollia
f7b24edf1e implement solved 2026-01-20 13:15:48 +01:00
michiel
e5ea838430 New CSV structure thats cleaner, the code in AITest is also cleaner 2026-01-19 13:05:17 +01:00
michiel
989e0a65c6 added visual score to reversi 2026-01-19 09:47:41 +01:00
Ticho Hidding
7565757b6b Research Experiment Data generator 2026-01-19 01:40:50 +01:00
Bas Antonius de Jong
a6b2356a5e update mcts, incremental merge (#311)
* mcts v1, v2, v3, v4 done. v5 wip

* update mcts

* mcts v1, v2, v3, v4 done. v5 wip

* update mcts

* Merge changes on dev

* update mcts

---------

Co-authored-by: ramollia <>
2026-01-17 04:05:11 +01:00
Bas Antonius de Jong
d078a70950 289 server (#308) Incremental server update, with working tournament and player input timeout
* Server update with new dev changes (#305)

* merge widgets with development

* readd previous game thread code

* Revert "readd previous game thread code"

This reverts commit d24feef73e.

* Revert "Merge remote-tracking branch 'origin/Development' into Development"

This reverts commit 59d46cb73c, reversing
changes made to 38681c5db0.

* Revert "merge widgets with development"

This reverts commit 38681c5db0.

* Merge 292 into development (#293)

Applied template method pattern to abstract player

* Added documentation to player classes and improved method names (#295)

* mcts v1

* bitboard optimization

* bitboard fix & mcts v2 & mcts v3. v3 still in progress and v4 coming soon

* main

---------

Co-authored-by: ramollia <>
Co-authored-by: Stef <stbuwalda@gmail.com>
Co-authored-by: Stef <48526421+StefBuwalda@users.noreply.github.com>

* Hotfix for stuff

* Logging and fixed user input getting stuck

* Fixed merge mistakes

* Working tournament

* GlobalEventBus is now async instead

* Shuffle now changeable, host can now switch tournament gametype

* Tournament results are now send back to the clients connected to the server

* Tournament now returns result to clients

* Refactored tournament to use interfaces and builders

* Removed unnecessary imports

* Tournament refactor for better naming and easier to understand code

* Starting a tournament now requires to be admin

* Request admin list

* Added admins to games

* Tournament is now without admins

* Added result comeback with a draw

* Async tournament runner

* Added back ability to shuffle matchmaker

* Moved scoring calculation into scoring system

* Tournament now uses propper builder pattern

* Null handling

* Removed input mistake, removed print

* Refactored Tournament to use matchExecutor and ResultBroadcaster. Added turnTime and players are now added through Tournament creation instead of on MatchMaker/ScoreSystem creation

* Added shuffle to builder

* Removed unnecessary throw

* More adaptable scoring system

* Moved async runner to virtual thread

* Timeout added

* AI player given time change

---------

Co-authored-by: Stef <stbuwalda@gmail.com>
Co-authored-by: Stef <48526421+StefBuwalda@users.noreply.github.com>
2026-01-16 13:06:09 +01:00
23 changed files with 1355 additions and 635 deletions

View File

@@ -1,53 +1,23 @@
package org.toop;
import org.toop.app.App;
import org.toop.framework.game.games.reversi.BitboardReversi;
import org.toop.framework.game.players.ArtificialPlayer;
import org.toop.game.players.ai.MCTSAI;
import org.toop.game.players.ai.RandomAI;
import org.toop.game.players.ai.mcts.MCTSAI1;
import org.toop.game.players.ai.mcts.MCTSAI2;
import org.toop.game.players.ai.mcts.MCTSAI3;
import org.toop.game.players.ai.mcts.MCTSAI4;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public final class Main {
static void main(String[] args) {
App.run(args);
// testMCTS(10);
// final ExecutorService executor = Executors.newFixedThreadPool(1);
// executor.execute(() -> testAIs(25));
}
// Voor onderzoek
// private static void testMCTS(int games) {
// var random = new ArtificialPlayer<>(new RandomAI<BitboardReversi>(), "Random AI");
// var v1 = new ArtificialPlayer<>(new MCTSAI<BitboardTicTacToe>(10), "MCTS V1 AI");
// var v2 = new ArtificialPlayer<>(new MCTSAI2<BitboardTicTacToe>(10), "MCTS V2 AI");
// var v2_2 = new ArtificialPlayer<>(new MCTSAI2<BitboardTicTacToe>(100), "MCTS V2_2 AI");
// var v3 = new ArtificialPlayer<>(new MCTSAI3<BitboardTicTacToe>(10), "MCTS V3 AI");
// testAI(games, new Player[]{ v1, v2 });
// // testAI(games, new Player[]{ v1, v3 });
// // testAI(games, new Player[]{ random, v3 });
// // testAI(games, new Player[]{ v2, v3 });
// testAI(games, new Player[]{ v2, v3 });
// // testAI(games, new Player[]{ v3, v2 });
// }
// private static void testAI(int games, Player<BitboardReversi>[] ais) {
// int wins = 0;
// int ties = 0;
// for (int i = 0; i < games; i++) {
// final BitboardReversi match = new BitboardReversi(ais);
// while (!match.isTerminal()) {
// final int currentAI = match.getCurrentTurn();
// final long move = ais[currentAI].getMove(match);
// match.play(move);
// }
// if (match.getWinner() < 0) {
// ties++;
// continue;
// }
// wins += match.getWinner() == 0? 1 : 0;
// }
// System.out.printf("Out of %d games, %s won %d -- tied %d -- lost %d, games against %s\n", games, ais[0].getName(), wins, ties, games - wins - ties, ais[1].getName());
// System.out.printf("Average win rate was: %.2f\n\n", wins / (float)games);
// }
}

View File

@@ -20,8 +20,7 @@ import org.toop.framework.networking.connection.clients.TournamentNetworkingClie
import org.toop.framework.networking.connection.events.NetworkEvents;
import org.toop.framework.networking.connection.types.NetworkingConnector;
import org.toop.framework.networking.server.gateway.NettyGatewayServer;
import org.toop.framework.game.players.LocalPlayer;
import org.toop.game.players.ai.MCTSAI3;
import org.toop.game.players.ai.mcts.MCTSAI3;
import org.toop.local.AppContext;
import java.util.Arrays;

View File

@@ -10,6 +10,7 @@ import org.toop.app.widget.WidgetContainer;
import org.toop.app.widget.view.GameView;
import org.toop.framework.eventbus.EventFlow;
import org.toop.framework.eventbus.GlobalEventBus;
import org.toop.framework.game.games.reversi.BitboardReversi;
import org.toop.framework.gameFramework.controller.GameController;
import org.toop.framework.gameFramework.model.game.threadBehaviour.SupportsOnlinePlay;
import org.toop.framework.gameFramework.model.game.TurnBasedGame;
@@ -158,8 +159,14 @@ public class GenericGameController implements GameController {
canvas.redraw(gameCopy);
String gameType = game.getClass().getSimpleName().replace("Bitboard","");
gameView.nextPlayer(true, getCurrentPlayer().getName(), game.getPlayer(1-getCurrentPlayerIndex()).getName(),gameType);
if (getCurrentPlayer() instanceof LocalPlayer && gameType.equals("Reversi")){
if (gameType.equals("Reversi")) {
BitboardReversi reversiGame = (BitboardReversi) game;
BitboardReversi.Score reversiScore = reversiGame.getScore();
gameView.setPlayer1Score(reversiScore.black());
gameView.setPlayer2Score(reversiScore.white());
if (getCurrentPlayer() instanceof LocalPlayer) {
((ReversiBitCanvas)canvas).drawLegalDots(gameCopy);
}
}
}
}

View File

@@ -11,13 +11,19 @@ import org.toop.framework.game.players.OnlinePlayer;
import java.util.Arrays;
public class ReversiBitController extends GenericGameController {
private BitboardReversi game;
public ReversiBitController(Player[] players) {
BitboardReversi game = new BitboardReversi();
game.init(players);
ThreadBehaviour thread = Arrays.stream(players).anyMatch(e -> e instanceof OnlinePlayer) ?
new OnlineThreadBehaviour(game) : new LocalThreadBehaviour(game);
super(new ReversiBitCanvas(), game, thread, "Reversi");
}
public BitboardReversi.Score getScore() {
return game.getScore();
}
}

View File

@@ -4,6 +4,7 @@ import org.toop.app.widget.complex.ConfirmWidget;
import org.toop.app.widget.complex.PopupWidget;
import javafx.geometry.Pos;
import org.toop.framework.game.games.reversi.BitboardReversi;
public final class GameOverPopup extends PopupWidget {
public GameOverPopup(boolean winOrTie, String winner) {
@@ -15,7 +16,6 @@ public final class GameOverPopup extends PopupWidget {
else{
confirmWidget.setMessage("It was a tie!");
}
confirmWidget.addButton("ok", this::hide);
add(Pos.CENTER, confirmWidget);

View File

@@ -26,6 +26,8 @@ public final class GameView extends ViewWidget {
private final Text player2Header;
private Circle player1Icon;
private Circle player2Icon;
private final Text player1Score;
private final Text player2Score;
private final Button forfeitButton;
private final Button exitButton;
private final TextField chatInput;
@@ -40,6 +42,8 @@ public final class GameView extends ViewWidget {
player2Header = Primitive.header("");
player1Icon = new Circle();
player2Icon = new Circle();
player1Score = Primitive.header("");
player2Score = Primitive.header("");
if (onForfeit != null) {
forfeitButton = Primitive.button("forfeit", () -> onForfeit.run(), false);
@@ -153,14 +157,16 @@ public final class GameView extends ViewWidget {
private void setPlayerInfoReversi() {
var player1box = Primitive.hbox(
player1Icon,
player1Header
player1Header,
player1Score
);
player1box.getStyleClass().add("hboxspacing");
var player2box = Primitive.hbox(
player2Icon,
player2Header
player2Header,
player2Score
);
player2box.getStyleClass().add("hboxspacing");
@@ -178,4 +184,12 @@ public final class GameView extends ViewWidget {
player2Icon.setFill(Color.BLACK);
add(Pos.TOP_RIGHT, playerInfo);
}
public void setPlayer1Score(int score) {
player1Score.setText("(" + Integer.toString(score) + ")");
}
public void setPlayer2Score(int score) {
player2Score.setText("(" + Integer.toString(score) + ")");
}
}

View File

@@ -4,6 +4,7 @@ import javafx.application.Platform;
import org.toop.app.GameInformation;
import org.toop.app.gameControllers.ReversiBitController;
import org.toop.app.gameControllers.TicTacToeBitController;
import org.toop.framework.game.players.LocalPlayer;
import org.toop.framework.gameFramework.controller.GameController;
import org.toop.framework.gameFramework.model.player.Player;
import org.toop.framework.game.players.ArtificialPlayer;
@@ -12,11 +13,10 @@ import org.toop.app.widget.complex.PlayerInfoWidget;
import org.toop.app.widget.complex.ViewWidget;
import org.toop.app.widget.popup.ErrorPopup;
import org.toop.app.widget.tutorial.*;
import org.toop.framework.game.players.LocalPlayer;
import org.toop.game.players.ai.MCTSAI;
import org.toop.game.players.ai.MCTSAI2;
import org.toop.game.players.ai.MCTSAI3;
import org.toop.game.players.ai.MiniMaxAI;
import org.toop.game.players.ai.mcts.MCTSAI1;
import org.toop.game.players.ai.mcts.MCTSAI3;
import org.toop.game.players.ai.mcts.MCTSAI4;
import org.toop.local.AppContext;
import javafx.geometry.Pos;
@@ -54,7 +54,7 @@ public class LocalMultiplayerView extends ViewWidget {
if (information.players[0].isHuman) {
players[0] = new LocalPlayer(information.players[0].name);
} else {
players[0] = new ArtificialPlayer(new MCTSAI(100), "MCTS AI");
players[0] = new ArtificialPlayer(new MCTSAI1(100), "MCTS AI");
}
if (information.players[1].isHuman) {
players[1] = new LocalPlayer(information.players[1].name);
@@ -82,13 +82,13 @@ public class LocalMultiplayerView extends ViewWidget {
if (information.players[0].isHuman) {
players[0] = new LocalPlayer(information.players[0].name);
} else {
// players[0] = new ArtificialPlayer(new RandomAI<BitboardReversi>(), "Random AI");
players[0] = new ArtificialPlayer(new MCTSAI3(50), "MCTS V3 AI");
// players[0] = new ArtificialPlayer(new RandomAI(), "Random AI");
players[0] = new ArtificialPlayer(new MCTSAI1(100), "MCTS V1 AI");
}
if (information.players[1].isHuman) {
players[1] = new LocalPlayer(information.players[1].name);
} else {
players[1] = new ArtificialPlayer(new MCTSAI(50), "MCTS V1 AI");
players[1] = new ArtificialPlayer(new MCTSAI4(100), "MCTS V4 AI");
}
if (AppSettings.getSettings().getTutorialFlag() && AppSettings.getSettings().getFirstReversi()) {
new ShowEnableTutorialWidget(

View File

@@ -17,8 +17,8 @@ public abstract class BitboardGame implements TurnBasedGame {
private Player[] players;
// long is 64 bits. Every game has a limit of 64 cells maximum.
private final long[] playerBitboard;
private int currentTurn = 0;
protected final long[] playerBitboard;
protected int currentTurn = 0;
private final int playerCount;
public BitboardGame(int columnSize, int rowSize, int playerCount) {
@@ -74,6 +74,8 @@ public abstract class BitboardGame implements TurnBasedGame {
return playerBitboard.length;
}
public int getAmountOfTurns() { return currentTurn; }
public int getCurrentTurn() {
return getCurrentPlayerIndex();
}

View File

@@ -4,7 +4,6 @@ import org.toop.framework.game.BitboardGame;
import org.toop.framework.gameFramework.GameState;
import org.toop.framework.gameFramework.model.game.PlayResult;
import org.toop.framework.gameFramework.model.player.Player;
import org.toop.framework.game.BitboardGame;
public class BitboardReversi extends BitboardGame {
@@ -321,8 +320,58 @@ public class BitboardReversi extends BitboardGame {
else if (blackCount > whiteCount){
return 0;
}
else{
else {
return 1;
}
}
@Override
public float rateMove(long move) {
final long corners = 0x8100000000000081L;
if ((move & corners) != 0L) {
return 0.4f;
}
final long xSquares = 0x0042000000004200L;
if ((move & xSquares) != 0) {
return -0.4f;
}
final long cSquares = 0x4281000000008142L;
if ((move & cSquares) != 0) {
return -0.1f;
}
return 0.0f;
}
@Override
public long heuristicMove(long legalMoves) {
long bestMove = 0L;
float bestMoveRate = Float.NEGATIVE_INFINITY;
while (legalMoves != 0L) {
final long move = legalMoves & -legalMoves;
final float moveRate = rateMove(move);
if (moveRate > bestMoveRate) {
bestMove = move;
bestMoveRate = moveRate;
}
legalMoves &= ~move;
}
return bestMove;
}
@Override
public void setFrom(long player1, long player2, int turn) {
this.playerBitboard[0] = player1;
this.playerBitboard[1] = player2;
this.currentTurn = turn;
}
}

View File

@@ -2,6 +2,7 @@ package org.toop.framework.game.games.tictactoe;
import org.toop.framework.gameFramework.GameState;
import org.toop.framework.gameFramework.model.game.PlayResult;
import org.toop.framework.gameFramework.model.game.TurnBasedGame;
import org.toop.framework.gameFramework.model.player.Player;
import org.toop.framework.game.BitboardGame;
@@ -110,4 +111,18 @@ public class BitboardTicTacToe extends BitboardGame {
public BitboardTicTacToe deepCopy() {
return new BitboardTicTacToe(this);
}
@Override
public float rateMove(long move) {
return 0.0f;
}
@Override
public long heuristicMove(long legalMoves) {
return legalMoves;
}
@Override
public void setFrom(long player1, long player2, int turn) {
}
}

View File

@@ -14,7 +14,9 @@ import org.toop.framework.gameFramework.model.game.TurnBasedGame;
*/
public class ArtificialPlayer extends AbstractPlayer {
/** The AI instance used to calculate moves. */
/**
* The AI instance used to calculate moves.
*/
private final AI ai;
/**
@@ -57,4 +59,8 @@ public class ArtificialPlayer extends AbstractPlayer {
public ArtificialPlayer deepCopy() {
return new ArtificialPlayer(this);
}
public AI getAi() {
return ai;
}
}

View File

@@ -13,4 +13,9 @@ public interface TurnBasedGame extends DeepCopyable<TurnBasedGame> {
PlayResult play(long move);
PlayResult getState();
boolean isTerminal();
float rateMove(long move);
long heuristicMove(long legalMoves);
void setFrom(long player1, long player2, int turn);
}

View File

@@ -1,193 +1,298 @@
package org.toop.game.players.ai;
import org.toop.framework.gameFramework.GameState;
import org.toop.framework.gameFramework.model.game.PlayResult;
import org.toop.framework.gameFramework.model.game.TurnBasedGame;
import org.toop.framework.gameFramework.model.player.AbstractAI;
import java.util.Random;
import java.util.concurrent.atomic.AtomicInteger;
public abstract class MCTSAI extends AbstractAI {
protected static class Node {
public static final int VIRTUAL_LOSS = -1;
public class MCTSAI extends AbstractAI {
private static class Node {
public TurnBasedGame state;
public long move;
public long unexpandedMoves;
public Node parent;
public int expanded;
public Node[] children;
public int visits;
public float value;
public AtomicInteger value;
public AtomicInteger visits;
public float heuristic;
public float solved;
public Node(TurnBasedGame state, Node parent, long move) {
final long legalMoves = state.getLegalMoves();
public Node(TurnBasedGame state, long move, Node parent) {
this.state = state;
this.move = move;
this.unexpandedMoves = legalMoves;
this.parent = parent;
this.children = new Node[Long.bitCount(legalMoves)];
this.expanded = 0;
this.children = new Node[Long.bitCount(state.getLegalMoves())];
this.value = new AtomicInteger(0);
this.visits = new AtomicInteger(0);
this.visits = 0;
this.value = 0.0f;
this.heuristic = state.rateMove(move);
this.solved = Float.NaN;
}
public Node(TurnBasedGame state) {
this(state, 0L, null);
this(state, null, 0L);
}
public int getExpanded() {
return children.length - Long.bitCount(unexpandedMoves);
}
public boolean isFullyExpanded() {
return expanded >= children.length;
return unexpandedMoves == 0L;
}
float calculateUCT() {
float exploitation = visits <= 0? 0 : value / visits;
float exploration = 1.41f * (float)(Math.sqrt(Math.log(visits) / visits));
public float calculateUCT(float explorationFactor) {
if (visits.get() == 0) {
return Float.POSITIVE_INFINITY;
}
return exploitation + exploration;
final float exploitation = (float) value.get() / visits.get();
final float exploration = (float)(Math.sqrt(explorationFactor / visits.get()));
final float bias = heuristic * 10.0f / (visits.get() + 1);
return exploitation + exploration + bias;
}
public Node bestUCTChild() {
int bestChildIndex = -1;
float bestScore = Float.NEGATIVE_INFINITY;
final int expanded = getExpanded();
Node highestUCTChild = null;
float highestUCT = Float.NEGATIVE_INFINITY;
for (int i = 0; i < expanded; i++) {
final float score = calculateUCT();
final float childUCT = children[i].calculateUCT(2.0f * (float)Math.log(visits.get()));
if (score > bestScore) {
bestChildIndex = i;
bestScore = score;
if (childUCT > highestUCT) {
highestUCTChild = children[i];
highestUCT = childUCT;
}
}
return bestChildIndex >= 0? children[bestChildIndex] : this;
return highestUCTChild;
}
}
private final int milliseconds;
protected static final ThreadLocal<Random> random = ThreadLocal.withInitial(Random::new);
protected final int milliseconds;
protected int lastIterations;
public MCTSAI(int milliseconds) {
this.milliseconds = milliseconds;
this.lastIterations = 0;
}
public MCTSAI(MCTSAI other) {
this.milliseconds = other.milliseconds;
this.lastIterations = other.lastIterations;
}
@Override
public MCTSAI deepCopy() {
return new MCTSAI(this);
public int getLastIterations() {
return lastIterations;
}
@Override
public long getMove(TurnBasedGame game) {
Node root = new Node(game.deepCopy());
protected Node selection(Node root) {
while (Float.isNaN(root.solved) && root.isFullyExpanded() && !root.state.isTerminal()) {
root.value.addAndGet(Node.VIRTUAL_LOSS);
root.visits.incrementAndGet();
long endTime = System.currentTimeMillis() + milliseconds;
while (System.currentTimeMillis() <= endTime) {
Node node = selection(root);
long legalMoves = node.state.getLegalMoves();
if (legalMoves != 0) {
node = expansion(node, legalMoves);
root = root.bestUCTChild();
}
float result = 0.0f;
root.value.addAndGet(Node.VIRTUAL_LOSS);
root.visits.incrementAndGet();
if (node.state.getLegalMoves() != 0) {
result = simulation(node.state, game.getCurrentTurn());
return root;
}
backPropagation(node, result);
protected Node expansion(Node leaf) {
synchronized (leaf) {
if (leaf.unexpandedMoves == 0L) {
return leaf;
}
int mostVisitedIndex = -1;
int mostVisits = -1;
final long unexpandedMove = leaf.unexpandedMoves & -leaf.unexpandedMoves;
for (int i = 0; i < root.expanded; i++) {
if (root.children[i].visits > mostVisits) {
mostVisitedIndex = i;
mostVisits = root.children[i].visits;
final TurnBasedGame copiedState = leaf.state.deepCopy();
copiedState.play(unexpandedMove);
final Node expandedChild = new Node(copiedState, leaf, unexpandedMove);
leaf.children[leaf.getExpanded()] = expandedChild;
leaf.unexpandedMoves &= ~unexpandedMove;
return expandedChild;
}
}
return mostVisitedIndex != -1? root.children[mostVisitedIndex].move : randomSetBit(game.getLegalMoves());
protected int simulation(Node leaf) {
final TurnBasedGame copiedState = leaf.state.deepCopy();
final int playerIndex = 1 - copiedState.getCurrentTurn();
while (!copiedState.isTerminal()) {
final long legalMoves = copiedState.getLegalMoves();
final long randomMove = randomSetBit(legalMoves);
copiedState.play(randomMove);
}
private Node selection(Node node) {
while (node.state.getLegalMoves() != 0L && node.isFullyExpanded()) {
node = node.bestUCTChild();
if (copiedState.getWinner() == playerIndex) {
return 1;
}
return node;
if (copiedState.getWinner() >= 0) {
return -1;
}
private Node expansion(Node node, long legalMoves) {
for (int i = 0; i < node.expanded; i++) {
legalMoves &= ~node.children[i].move;
return 0;
}
if (legalMoves == 0L) {
return node;
protected void backPropagation(Node leaf, int value) {
while (leaf != null) {
value -= Node.VIRTUAL_LOSS;
leaf.value.addAndGet(value);
if (Float.isNaN(leaf.solved)) {
updateSolvedStatus(leaf);
}
long move = randomSetBit(legalMoves);
TurnBasedGame copy = node.state.deepCopy();
copy.play(move);
Node newlyExpanded = new Node(copy, move, node);
node.children[node.expanded] = newlyExpanded;
node.expanded++;
return newlyExpanded;
}
private float simulation(TurnBasedGame state, int playerIndex) {
TurnBasedGame copy = state.deepCopy();
long legalMoves = copy.getLegalMoves();
PlayResult result = null;
while (legalMoves != 0) {
result = copy.play(randomSetBit(legalMoves));
legalMoves = copy.getLegalMoves();
}
if (result.state() == GameState.WIN) {
if (result.player() == playerIndex) {
return 1.0f;
}
return -1.0f;
}
return -0.2f;
}
private void backPropagation(Node node, float value) {
while (node != null) {
node.visits++;
node.value += value;
node = node.parent;
value = -value;
leaf = leaf.parent;
}
}
public static long randomSetBit(long value) {
Random random = new Random();
protected Node mostVisitedChild(Node root) {
final int expanded = root.getExpanded();
int count = Long.bitCount(value);
int target = random.nextInt(count);
Node mostVisitedChild = null;
int mostVisited = -1;
while (true) {
int bit = Long.numberOfTrailingZeros(value);
if (target == 0) {
return 1L << bit;
for (int i = 0; i < expanded; i++) {
if (root.children[i].visits.get() > mostVisited) {
mostVisitedChild = root.children[i];
mostVisited = root.children[i].visits.get();
}
}
return mostVisitedChild;
}
protected Node findOrResetRoot(Node root, TurnBasedGame game) {
if (root == null) {
return new Node(game.deepCopy());
}
if (areStatesEqual(root.state.getBoard(), game.getBoard())) {
return root;
}
final int expanded = root.getExpanded();
for (int i = 0; i < expanded; i++) {
if (areStatesEqual(root.children[i].state.getBoard(), game.getBoard())) {
root.children[i].parent = null;
return root.children[i];
}
}
return new Node(game.deepCopy());
}
protected Node findChildByMove(Node root, long move) {
final int expanded = root.getExpanded();
for (int i = 0; i < expanded; i++) {
if (root.children[i].move == move) {
root.children[i].parent = null;
return root.children[i];
}
}
return null;
}
protected boolean areStatesEqual(long[] state1, long[] state2) {
if (state1.length != state2.length) {
return false;
}
for (int i = 0; i < state1.length; i++) {
if (state1[i] != state2[i]) {
return false;
}
}
return true;
}
protected long randomSetBit(long value) {
if (0L == value) {
return 0;
}
final int bitCount = Long.bitCount(value);
final int randomBitCount = random.get().nextInt(bitCount);
for (int i = 0; i < randomBitCount; i++) {
value &= value - 1;
target--;
}
return value & -value;
}
private void updateSolvedStatus(Node node) {
if (node.state.isTerminal()) {
final int winner = node.state.getWinner();
final int mover = 1 - node.state.getCurrentTurn();
node.solved = winner == mover? 1.0f : winner == -1? 0.0f : -1.0f;
return;
}
if (node.isFullyExpanded()) {
boolean allChildrenSolved = true;
boolean foundWinningMove = false;
boolean foundDrawMove = false;
for (final Node child : node.children) {
if (!Float.isNaN(child.solved)) {
if (child.solved == -1.0f) {
foundWinningMove = true;
break;
}
if (child.solved == 0.0f) {
foundDrawMove = true;
}
} else {
allChildrenSolved = false;
}
}
if (foundWinningMove) {
node.solved = 1.0f;
} else if (allChildrenSolved) {
node.solved = foundDrawMove? 0.0f : -1.0f;
}
}
}
}

View File

@@ -1,195 +0,0 @@
package org.toop.game.players.ai;
import org.toop.framework.gameFramework.model.game.TurnBasedGame;
import org.toop.framework.gameFramework.model.player.AbstractAI;
import java.util.Random;
public class MCTSAI2 extends AbstractAI {
private static class Node {
public TurnBasedGame state;
public long move;
public long unexpandedMoves;
public Node parent;
public Node[] children;
public int expanded;
public float value;
public int visits;
public Node(TurnBasedGame state, Node parent, long move) {
final long legalMoves = state.getLegalMoves();
this.state = state;
this.move = move;
this.unexpandedMoves = legalMoves;
this.parent = parent;
this.children = new Node[Long.bitCount(legalMoves)];
this.expanded = 0;
this.value = 0.0f;
this.visits = 0;
}
public Node(TurnBasedGame state) {
this(state, null, 0L);
}
public boolean isFullyExpanded() {
return expanded == children.length;
}
public float calculateUCT(int parentVisits) {
final float exploitation = value / visits;
final float exploration = 1.41f * (float)(Math.sqrt(Math.log(parentVisits) / visits));
return exploitation + exploration;
}
public Node bestUCTChild() {
Node highestUCTChild = null;
float highestUCT = Float.NEGATIVE_INFINITY;
for (int i = 0; i < expanded; i++) {
final float childUCT = children[i].calculateUCT(visits);
if (childUCT > highestUCT) {
highestUCTChild = children[i];
highestUCT = childUCT;
}
}
return highestUCTChild;
}
}
private final Random random;
private final int milliseconds;
public MCTSAI2(int milliseconds) {
this.random = new Random();
this.milliseconds = milliseconds;
}
public MCTSAI2(MCTSAI2 other) {
this.random = other.random;
this.milliseconds = other.milliseconds;
}
@Override
public MCTSAI2 deepCopy() {
return new MCTSAI2(this);
}
@Override
public long getMove(TurnBasedGame game) {
final Node root = new Node(game, null, 0L);
final long endTime = System.nanoTime() + milliseconds * 1_000_000L;
while (System.nanoTime() < endTime) {
Node leaf = selection(root);
leaf = expansion(leaf);
final float value = simulation(leaf);
backPropagation(leaf, value);
}
final Node mostVisitedChild = mostVisitedChild(root);
return mostVisitedChild != null? mostVisitedChild.move : 0L;
}
private Node mostVisitedChild(Node root) {
Node mostVisitedChild = null;
int mostVisited = -1;
for (int i = 0; i < root.expanded; i++) {
if (root.children[i].visits > mostVisited) {
mostVisitedChild = root.children[i];
mostVisited = root.children[i].visits;
}
}
return mostVisitedChild;
}
private Node selection(Node root) {
while (root.isFullyExpanded() && !root.state.isTerminal()) {
root = root.bestUCTChild();
}
return root;
}
private Node expansion(Node leaf) {
if (leaf.unexpandedMoves == 0L) {
return leaf;
}
final long unexpandedMove = leaf.unexpandedMoves & -leaf.unexpandedMoves;
final TurnBasedGame copiedState = leaf.state.deepCopy();
copiedState.play(unexpandedMove);
final Node expandedChild = new Node(copiedState, leaf, unexpandedMove);
leaf.children[leaf.expanded] = expandedChild;
leaf.expanded++;
leaf.unexpandedMoves &= ~unexpandedMove;
return expandedChild;
}
private float simulation(Node leaf) {
final TurnBasedGame copiedState = leaf.state.deepCopy();
final int playerIndex = 1 - copiedState.getCurrentTurn();
while (!copiedState.isTerminal()) {
final long legalMoves = copiedState.getLegalMoves();
final long randomMove = randomSetBit(legalMoves);
copiedState.play(randomMove);
}
if (copiedState.getWinner() == playerIndex) {
return 1.0f;
} else if (copiedState.getWinner() >= 0) {
return -1.0f;
}
return 0.0f;
}
private void backPropagation(Node leaf, float value) {
while (leaf != null) {
leaf.value += value;
leaf.visits++;
value = -value;
leaf = leaf.parent;
}
}
private long randomSetBit(long value) {
if (0L == value) {
return 0;
}
final int bitCount = Long.bitCount(value);
final int randomBitCount = random.nextInt(bitCount);
for (int i = 0; i < randomBitCount; i++) {
value &= value - 1;
}
return value & -value;
}
}

View File

@@ -1,258 +0,0 @@
package org.toop.game.players.ai;
import org.toop.framework.gameFramework.model.game.TurnBasedGame;
import org.toop.framework.gameFramework.model.player.AbstractAI;
import java.util.Random;
public class MCTSAI3 extends AbstractAI {
private static class Node {
public TurnBasedGame state;
public long move;
public long unexpandedMoves;
public Node parent;
public Node[] children;
public int expanded;
public float value;
public int visits;
public Node(TurnBasedGame state, Node parent, long move) {
final long legalMoves = state.getLegalMoves();
this.state = state;
this.move = move;
this.unexpandedMoves = legalMoves;
this.parent = parent;
this.children = new Node[Long.bitCount(legalMoves)];
this.expanded = 0;
this.value = 0.0f;
this.visits = 0;
}
public Node(TurnBasedGame state) {
this(state, null, 0L);
}
public boolean isFullyExpanded() {
return expanded == children.length;
}
public float calculateUCT(int parentVisits) {
final float exploitation = value / visits;
final float exploration = 1.41f * (float)(Math.sqrt(Math.log(parentVisits) / visits));
return exploitation + exploration;
}
public Node bestUCTChild() {
Node highestUCTChild = null;
float highestUCT = Float.NEGATIVE_INFINITY;
for (int i = 0; i < expanded; i++) {
final float childUCT = children[i].calculateUCT(visits);
if (childUCT > highestUCT) {
highestUCTChild = children[i];
highestUCT = childUCT;
}
}
return highestUCTChild;
}
}
private final Random random;
private Node root;
private final int milliseconds;
public MCTSAI3(int milliseconds) {
this.random = new Random();
this.root = null;
this.milliseconds = milliseconds;
}
public MCTSAI3(MCTSAI3 other) {
this.random = other.random;
this.root = other.root;
this.milliseconds = other.milliseconds;
}
@Override
public MCTSAI3 deepCopy() {
return new MCTSAI3(this);
}
@Override
public long getMove(TurnBasedGame game) {
detectRoot(game);
final long endTime = System.nanoTime() + milliseconds * 1_000_000L;
while (System.nanoTime() < endTime) {
Node leaf = selection(root);
leaf = expansion(leaf);
final float value = simulation(leaf);
backPropagation(leaf, value);
}
final Node mostVisitedChild = mostVisitedChild(root);
final long move = mostVisitedChild != null? mostVisitedChild.move : 0L;
newRoot(move);
return move;
}
private Node mostVisitedChild(Node root) {
Node mostVisitedChild = null;
int mostVisited = -1;
for (int i = 0; i < root.expanded; i++) {
if (root.children[i].visits > mostVisited) {
mostVisitedChild = root.children[i];
mostVisited = root.children[i].visits;
}
}
return mostVisitedChild;
}
private void detectRoot(TurnBasedGame game) {
if (root == null) {
root = new Node(game.deepCopy());
return;
}
final long[] currentBoards = game.getBoard();
final long[] rootBoards = root.state.getBoard();
boolean detected = true;
for (int i = 0; i < rootBoards.length; i++) {
if (rootBoards[i] != currentBoards[i]) {
detected = false;
break;
}
}
if (detected) {
return;
}
for (int i = 0; i < root.expanded; i++) {
final Node child = root.children[i];
final long[] childBoards = child.state.getBoard();
detected = true;
for (int j = 0; j < childBoards.length; j++) {
if (childBoards[j] != currentBoards[j]) {
detected = false;
break;
}
}
if (detected) {
root = child;
return;
}
}
root = new Node(game.deepCopy());
}
private void newRoot(long move) {
for (final Node child : root.children) {
if (child.move == move) {
root = child;
break;
}
}
}
private Node selection(Node root) {
while (root.isFullyExpanded() && !root.state.isTerminal()) {
root = root.bestUCTChild();
}
return root;
}
private Node expansion(Node leaf) {
if (leaf.unexpandedMoves == 0L) {
return leaf;
}
final long unexpandedMove = leaf.unexpandedMoves & -leaf.unexpandedMoves;
final TurnBasedGame copiedState = leaf.state.deepCopy();
copiedState.play(unexpandedMove);
final Node expandedChild = new Node(copiedState, leaf, unexpandedMove);
leaf.children[leaf.expanded] = expandedChild;
leaf.expanded++;
leaf.unexpandedMoves &= ~unexpandedMove;
return expandedChild;
}
private float simulation(Node leaf) {
final TurnBasedGame copiedState = leaf.state.deepCopy();
final int playerIndex = 1 - copiedState.getCurrentTurn();
while (!copiedState.isTerminal()) {
final long legalMoves = copiedState.getLegalMoves();
final long randomMove = randomSetBit(legalMoves);
copiedState.play(randomMove);
}
if (copiedState.getWinner() == playerIndex) {
return 1.0f;
} else if (copiedState.getWinner() >= 0) {
return -1.0f;
}
return 0.0f;
}
private void backPropagation(Node leaf, float value) {
while (leaf != null) {
leaf.value += value;
leaf.visits++;
value = -value;
leaf = leaf.parent;
}
}
private long randomSetBit(long value) {
if (0L == value) {
return 0;
}
final int bitCount = Long.bitCount(value);
final int randomBitCount = random.nextInt(bitCount);
for (int i = 0; i < randomBitCount; i++) {
value &= value - 1;
}
return value & -value;
}
}

View File

@@ -0,0 +1,38 @@
package org.toop.game.players.ai.mcts;
import org.toop.framework.gameFramework.model.game.TurnBasedGame;
import org.toop.game.players.ai.MCTSAI;
public class MCTSAI1 extends MCTSAI {
public MCTSAI1(int milliseconds) {
super(milliseconds);
}
public MCTSAI1(MCTSAI1 other) {
super(other);
}
@Override
public MCTSAI1 deepCopy() {
return new MCTSAI1(this);
}
@Override
public long getMove(TurnBasedGame game) {
final Node root = new Node(game, null, 0L);
final long endTime = System.nanoTime() + milliseconds * 1_000_000L;
while (Float.isNaN(root.solved) && System.nanoTime() < endTime) {
Node leaf = selection(root);
leaf = expansion(leaf);
final int value = simulation(leaf);
backPropagation(leaf, value);
}
lastIterations = root.visits.get();
final Node mostVisitedChild = mostVisitedChild(root);
return mostVisitedChild.move;
}
}

View File

@@ -0,0 +1,48 @@
package org.toop.game.players.ai.mcts;
import org.toop.framework.gameFramework.model.game.TurnBasedGame;
import org.toop.game.players.ai.MCTSAI;
public class MCTSAI2 extends MCTSAI {
private Node root;
public MCTSAI2(int milliseconds) {
super(milliseconds);
this.root = null;
}
public MCTSAI2(MCTSAI2 other) {
super(other);
this.root = other.root;
}
@Override
public MCTSAI2 deepCopy() {
return new MCTSAI2(this);
}
@Override
public long getMove(TurnBasedGame game) {
root = findOrResetRoot(root, game);
final long endTime = System.nanoTime() + milliseconds * 1_000_000L;
while (Float.isNaN(root.solved) && System.nanoTime() < endTime) {
Node leaf = selection(root);
leaf = expansion(leaf);
final int value = simulation(leaf);
backPropagation(leaf, value);
}
lastIterations = root.visits.get();
final Node mostVisitedChild = mostVisitedChild(root);
final long move = mostVisitedChild.move;
root = findChildByMove(root, move);
return move;
}
}

View File

@@ -0,0 +1,76 @@
package org.toop.game.players.ai.mcts;
import org.toop.framework.gameFramework.model.game.TurnBasedGame;
import org.toop.game.players.ai.MCTSAI;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
public class MCTSAI3 extends MCTSAI {
private final int threads;
private final ExecutorService threadPool;
public MCTSAI3(int milliseconds, int threads) {
super(milliseconds);
this.threads = threads;
this.threadPool = Executors.newFixedThreadPool(threads);
}
public MCTSAI3(MCTSAI3 other) {
super(other);
this.threads = other.threads;
this.threadPool = other.threadPool;
}
@Override
public MCTSAI3 deepCopy() {
return new MCTSAI3(this);
}
@Override
public long getMove(TurnBasedGame game) {
final Node root = new Node(game.deepCopy(), null, 0L);
final long endTime = System.nanoTime() + milliseconds * 1_000_000L;
final CountDownLatch latch = new CountDownLatch(threads);
for (int i = 0; i < threads; i++) {
threadPool.submit(() -> {
try {
iterate(root, endTime);
} finally {
latch.countDown();
}
});
}
try {
final long remaining = endTime - System.nanoTime();
latch.await(remaining, TimeUnit.NANOSECONDS);
lastIterations = root.visits.get();
final Node mostVisitedChild = mostVisitedChild(root);
return mostVisitedChild.move;
} catch (Exception _) {
lastIterations = 0;
final long legalMoves = game.getLegalMoves();
return randomSetBit(legalMoves);
}
}
private void iterate(Node root, long endTime) {
while (Float.isNaN(root.solved) && System.nanoTime() < endTime) {
Node leaf = selection(root);
leaf = expansion(leaf);
final int value = simulation(leaf);
backPropagation(leaf, value);
}
}
}

View File

@@ -0,0 +1,82 @@
package org.toop.game.players.ai.mcts;
import org.toop.framework.gameFramework.model.game.TurnBasedGame;
import org.toop.game.players.ai.MCTSAI;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
public class MCTSAI4 extends MCTSAI {
private final int threads;
private final ExecutorService threadPool;
private Node root;
public MCTSAI4(int milliseconds, int threads) {
super(milliseconds);
this.threads = threads;
this.threadPool = Executors.newFixedThreadPool(threads);
}
public MCTSAI4(MCTSAI4 other) {
super(other);
this.threads = other.threads;
this.threadPool = other.threadPool;
}
@Override
public MCTSAI4 deepCopy() {
return new MCTSAI4(this);
}
@Override
public long getMove(TurnBasedGame game) {
root = findOrResetRoot(root, game);
final long endTime = System.nanoTime() + milliseconds * 1_000_000L;
final CountDownLatch latch = new CountDownLatch(threads);
for (int i = 0; i < threads; i++) {
threadPool.submit(() -> {
try {
iterate(root, endTime);
} finally {
latch.countDown();
}
});
}
try {
final long remaining = endTime - System.nanoTime();
latch.await(remaining, TimeUnit.NANOSECONDS);
lastIterations = root.visits.get();
final Node mostVisitedChild = mostVisitedChild(root);
final long move = mostVisitedChild.move;
root = findChildByMove(root, move);
return move;
} catch (Exception _) {
lastIterations = 0;
final long legalMoves = game.getLegalMoves();
return randomSetBit(legalMoves);
}
}
private void iterate(Node root, long endTime) {
while (Float.isNaN(root.solved) && System.nanoTime() < endTime) {
Node leaf = selection(root);
leaf = expansion(leaf);
final int value = simulation(leaf);
backPropagation(leaf, value);
}
}
}

View File

@@ -0,0 +1,77 @@
package research;
public class AIData {
public String AI;
public long gamesPlayed;
public double winrate;
public double averageIterations;
public double averageIterations10;
public double averageIterations20;
public double averageIterations30;
public AIData(String AI, long gamesPlayed, double winrate, double averageIterations, double averageIterations10, double averageIterations20, double averageIterations30) {
this.AI = AI;
this.gamesPlayed = gamesPlayed;
this.winrate = winrate;
this.averageIterations = averageIterations;
this.averageIterations10 = averageIterations10;
this.averageIterations20 = averageIterations20;
this.averageIterations30 = averageIterations30;
}
public String getAI() {
return AI;
}
public void setAI(String AI) {
this.AI = AI;
}
public long getGamesPlayed() {
return gamesPlayed;
}
public void setGamesPlayed(long gamesPlayed) {
this.gamesPlayed = gamesPlayed;
}
public double getWinrate() {
return winrate;
}
public void setWinrate(double winrate) {
this.winrate = winrate;
}
public double getAverageIterations() {
return averageIterations;
}
public void setAverageIterations(double averageIterations) {
this.averageIterations = averageIterations;
}
public double getAverageIterations10() {
return averageIterations10;
}
public void setAverageIterations10(double averageIterations10) {
this.averageIterations10 = averageIterations10;
}
public double getAverageIterations20() {
return averageIterations20;
}
public void setAverageIterations20(double averageIterations20) {
this.averageIterations20 = averageIterations20;
}
public double getAverageIterations30() {
return averageIterations30;
}
public void setAverageIterations30(double averageIterations30) {
this.averageIterations30 = averageIterations30;
}
}

View File

@@ -0,0 +1,612 @@
package research;
import org.apache.maven.surefire.shared.io.FileDeleteStrategy;
import org.junit.jupiter.api.*;
import org.toop.framework.game.games.reversi.BitboardReversi;
import org.toop.framework.game.players.ArtificialPlayer;
import org.toop.game.players.ai.MCTSAI;
import org.toop.game.players.ai.mcts.MCTSAI1;
import org.toop.game.players.ai.mcts.MCTSAI2;
import org.toop.game.players.ai.mcts.MCTSAI3;
import org.toop.game.players.ai.mcts.MCTSAI4;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.nio.file.StandardOpenOption;
import java.time.LocalTime;
import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.List;
public class AITest {
private static String fileName = "gameData.csv";
private static List<Matchup> matchupList = new ArrayList<Matchup>();
private static List<AIData> dataList = new ArrayList<AIData>();
private static List<GameData> gameDataList = new ArrayList<GameData>();
@BeforeAll
public static void init() {
var versions = new ArtificialPlayer[4];
versions[0] = new ArtificialPlayer(new MCTSAI1(10), "MCTS V1");
versions[1] = new ArtificialPlayer(new MCTSAI2(10), "MCTS V2");
versions[2] = new ArtificialPlayer(new MCTSAI3(10, 8), "MCTS V3");
versions[3] = new ArtificialPlayer(new MCTSAI4(10, 8), "MCTS V4");
for (int i = 0; i < versions.length; i++) {
for (int j = i + 1; j < versions.length; j++) {
final int playerIndex1 = i % versions.length;
final int playerIndex2 = j % versions.length;
addMatch(versions[playerIndex1], versions[playerIndex2]);
addMatch(versions[playerIndex2], versions[playerIndex1]); // home vs away system
}
}
}
// @BeforeAll
// public static void init() {
//
// var versions = new ArtificialPlayer[11];
// versions[0] = new ArtificialPlayer(new MCTSAI3(10, 1), "MCTS V3T1");
// versions[1] = new ArtificialPlayer(new MCTSAI3(10, 2), "MCTS V3T2");
// versions[2] = new ArtificialPlayer(new MCTSAI3(10, 4), "MCTS V3T4");
// versions[3] = new ArtificialPlayer(new MCTSAI3(10, 8), "MCTS V3T8");
// versions[4] = new ArtificialPlayer(new MCTSAI3(10, 16), "MCTS V3T16");
// versions[5] = new ArtificialPlayer(new MCTSAI3(10, 128), "MCTS V3T32");
// versions[6] = new ArtificialPlayer(new MCTSAI3(10, 256), "MCTS V3T64");
// versions[7] = new ArtificialPlayer(new MCTSAI3(10, 128), "MCTS V3T128");
// versions[8] = new ArtificialPlayer(new MCTSAI3(10, 256), "MCTS V3T256");
// versions[9] = new ArtificialPlayer(new MCTSAI3(10, 512), "MCTS V3T512");
// versions[10] = new ArtificialPlayer(new MCTSAI3(10, 1024), "MCTS V3T1024");
//
// for (int i = 0; i < versions.length; i++) {
// for (int j = i + 1; j < versions.length; j++) {
// final int playerIndex1 = i % versions.length;
// final int playerIndex2 = j % versions.length;
// addMatch(versions[playerIndex1], versions[playerIndex2]);
// addMatch(versions[playerIndex2], versions[playerIndex1]); // home vs away system
// }
// }
// }
public static void addMatch(ArtificialPlayer v1, ArtificialPlayer v2) {
matchupList.add(new Matchup(v1, v2));
}
public void addAIData(AIData data) {
dataList.add(data);
}
public void addGameData(GameData data) {
gameDataList.add(data);
}
@Test
public void testAIvsAI() {
while (true) {
for (Matchup m : matchupList) {
playGame(m);
}
}
}
public void playGame(Matchup m) {
long nanocounterAI1 = 0L;
long nanocounterAI2 = 0L;
List<Integer> iterationsAI1 = new ArrayList<>();
List<Integer> iterationsAI2 = new ArrayList<>();
final BitboardReversi match = new BitboardReversi();
ArtificialPlayer[] players = new ArtificialPlayer[2];
players[0] = m.getPlayer1();
players[1] = m.getPlayer2();
match.init(players);
while (!match.isTerminal()) {
final int currentAI = match.getCurrentTurn();
final long startTime = System.nanoTime();
final long move = players[currentAI].getMove(match);
final long endTime = System.nanoTime();
if (players[currentAI].getAi() instanceof MCTSAI) {
final int lastIterations = ((MCTSAI) players[currentAI].getAi()).getLastIterations();
if (currentAI == 0) {
iterationsAI1.add(lastIterations);
nanocounterAI1 += (endTime - startTime);
} else {
iterationsAI2.add(lastIterations);
nanocounterAI2 += (endTime - startTime);
}
}
match.play(move);
}
generateMatchData(m.getPlayer1().getName(), m.getPlayer2().getName(), match, iterationsAI1, iterationsAI2, nanocounterAI1, nanocounterAI2);
}
public void generateMatchData(
String AI1,
String AI2,
BitboardReversi match,
List<Integer> iterationsAI1,
List<Integer> iterationsAI2,
long nanocounterAI1,
long nanocounterAI2
) {
try {
var ai110 = iterationsAI1.subList(0, 10);
var ai120 = iterationsAI1.subList(10, 20);
var ai130 = iterationsAI1.subList(20, iterationsAI1.size());
var ai210 = iterationsAI2.subList(0, 10);
var ai220 = iterationsAI2.subList(10, 20);
var ai230 = iterationsAI2.subList(20, iterationsAI2.size());
writeGamesToCSV(fileName, new GameData(
AI1,
AI2,
getWinnerForMatch(AI1, AI2, match),
match.getAmountOfTurns(),
iterationsAI1.stream().mapToLong(Integer::longValue).sum(),
ai110.stream().mapToLong(Integer::longValue).sum(),
ai120.stream().mapToLong(Integer::longValue).sum(),
ai130.stream().mapToLong(Integer::longValue).sum(),
iterationsAI1.stream().mapToDouble(Integer::doubleValue).sum() / iterationsAI1.size(),
ai110.stream().mapToDouble(Integer::doubleValue).sum() / ai110.size(),
ai120.stream().mapToDouble(Integer::doubleValue).sum() / ai120.size(),
ai130.stream().mapToDouble(Integer::doubleValue).sum() / ai130.size(),
iterationsAI2.stream().mapToInt(Integer::intValue).sum(),
ai210.stream().mapToLong(Integer::longValue).sum(),
ai220.stream().mapToLong(Integer::longValue).sum(),
ai230.stream().mapToLong(Integer::longValue).sum(),
iterationsAI2.stream().mapToDouble(Integer::doubleValue).sum() / iterationsAI2.size(),
ai210.stream().mapToDouble(Integer::doubleValue).sum() / ai210.size(),
ai220.stream().mapToDouble(Integer::doubleValue).sum() / ai220.size(),
ai230.stream().mapToDouble(Integer::doubleValue).sum() / ai230.size(),
nanocounterAI1,
nanocounterAI2,
LocalTime.now().format(DateTimeFormatter.ofPattern("HH:mm:ss"))
));
} catch (IOException e) {
throw new RuntimeException(e);
} catch (IndexOutOfBoundsException e) {
return;
}
}
public String getWinnerForMatch(String AI1, String AI2, BitboardReversi match) {
if (match.getWinner() == 0) {
return AI1;
}
if (match.getWinner() == 1) {
return AI2;
} else {
return "TIE";
}
}
public void generateData(Matchup matchup, BitboardReversi match, List<Integer> iterationsAI1, List<Integer> iterationsAI2) {
boolean matchup1Found = false;
boolean matchup2Found = false;
for (AIData aiData : dataList) {
if (aiData.getAI().equals(matchup.getPlayer1().getName())) {
matchup1Found = true;
} if (aiData.getAI().equals(matchup.getPlayer2().getName())) {
matchup2Found = true;
}
}
if (!(matchup1Found)) {
addAIData(new AIData(matchup.getPlayer1().getName(), 0, 0, 0, 0, 0, 0));
}
if (!(matchup2Found)) {
addAIData(new AIData(matchup.getPlayer2().getName(), 0, 0, 0, 0, 0, 0));
}
for (AIData aiData : dataList) { // set data for player 1
if (aiData.getAI().equals(matchup.getPlayer1().getName())) {
aiData.setGamesPlayed(aiData.getGamesPlayed() + 1);
aiData.setWinrate(calculateWinrate(0, aiData.getWinrate(), aiData.getGamesPlayed(), match.getWinner()));
aiData.setAverageIterations(calculateAverageIterations(aiData.getAverageIterations(), iterationsAI1));
aiData.setAverageIterations10(calculateAverageIterationsStartEnd(0, 10, aiData.getAverageIterations10(), iterationsAI1));
aiData.setAverageIterations20(calculateAverageIterationsStartEnd(10, 20, aiData.getAverageIterations20(), iterationsAI1));
aiData.setAverageIterations30(calculateAverageIterationsStartEnd(20, iterationsAI1.size(), aiData.getAverageIterations30(), iterationsAI1));
}
}
for (AIData aiData : dataList) {
if (aiData.getAI().equals(matchup.getPlayer2().getName())) {
aiData.setGamesPlayed(aiData.getGamesPlayed() + 1);
aiData.setWinrate(calculateWinrate(1, aiData.getWinrate(), aiData.getGamesPlayed(), match.getWinner()));
aiData.setAverageIterations(calculateAverageIterations(aiData.getAverageIterations(), iterationsAI2));
aiData.setAverageIterations10(calculateAverageIterationsStartEnd(0, 10, aiData.getAverageIterations10(), iterationsAI2));
aiData.setAverageIterations20(calculateAverageIterationsStartEnd(10, 20, aiData.getAverageIterations20(), iterationsAI2));
aiData.setAverageIterations30(calculateAverageIterationsStartEnd(20, iterationsAI2.size(), aiData.getAverageIterations30(), iterationsAI2));
}
}
}
public double calculateWinrate(int player, double winrate, long gamesPlayed, int winner) {
double result;
if (winner == 0 && player == 0 || winner == 1 && player == 1) {
return (winrate * (gamesPlayed - 1) + 1) / gamesPlayed;
} else if (winner == 0 && player == 1 || winner == 1 && player == 0) {
return (winrate * (gamesPlayed - 1) + 0) / gamesPlayed;
}
return (winrate * (gamesPlayed - 1) + 0) / gamesPlayed;
}
public double calculateAverageIterations(double averageIterations, List<Integer> thisGameIterations) {
double thisGameIterationsAverage = 0;
for (int iterations = 0; iterations < thisGameIterations.size(); iterations += 1) {
thisGameIterationsAverage += thisGameIterations.get(iterations);
}
thisGameIterationsAverage /= thisGameIterations.size();
return (averageIterations + thisGameIterationsAverage) / 2;
}
public double calculateAverageIterationsStartEnd(int start, int end, double averageIterations, List<Integer> thisGameIterations) {
double thisGameIterationsAverage = 0;
for (int iterations = start; iterations < end; iterations += 1) {
thisGameIterationsAverage += thisGameIterations.get(iterations);
}
thisGameIterationsAverage /= (end - start);
return (averageIterations + thisGameIterationsAverage) / 2;
}
@AfterAll
public static void writeAfterTests() {
try {
writeAIToCsv("Data.csv", dataList);
} catch (IOException e) {
e.printStackTrace();
}
}
public static void writeGamesToCSV(String filepath, GameData gameData) throws IOException {
try (
final BufferedWriter writer = Files.newBufferedWriter(
Paths.get(filepath),
StandardCharsets.UTF_8,
StandardOpenOption.CREATE,
StandardOpenOption.APPEND
);
final BufferedReader reader = new BufferedReader(new FileReader(filepath))
) {
if (reader.readLine() == null || reader.readLine().isBlank()) {
writer.write("Black,White,Winner,Turns Played,Black total iterations,Black total iterations 0-10,Black total iterations 11-20,Black total iterations 21-30,Black average iterations,Black average iterations 0-10,Black average iterations 11-20,Black average iterations 21-30,White total iterations,White total iterations 0-10,White total iterations 11-20,White total iterations 21-30,White average iterations,White average iterations 0-10,White average iterations 11-20,White average iterations 21-30,Total Time AI1,Total Time AI2,Time");
writer.newLine();
}
writer.write(
gameData.AI1() + "," +
gameData.AI2() + "," +
gameData.winner() + "," +
gameData.turns() + "," +
gameData.AI1totalIterations() + "," +
gameData.AI1totalIterations10() + "," +
gameData.AI1totalIterations20() + "," +
gameData.AI1totalIterations30() + "," +
BigDecimal.valueOf(gameData.AI1averageIterations()).setScale(2, RoundingMode.HALF_EVEN) + "," +
BigDecimal.valueOf(gameData.AI1averageIterations10()).setScale(2, RoundingMode.HALF_EVEN) + "," +
BigDecimal.valueOf(gameData.AI1averageIterations20()).setScale(2, RoundingMode.HALF_EVEN) + "," +
BigDecimal.valueOf(gameData.AI1averageIterations30()).setScale(2, RoundingMode.HALF_EVEN) + "," +
gameData.AI2totalIterations() + "," +
gameData.AI2totalIterations10() + "," +
gameData.AI2totalIterations20() + "," +
gameData.AI2totalIterations30() + "," +
BigDecimal.valueOf(gameData.AI2averageIterations()).setScale(2, RoundingMode.HALF_EVEN) + "," +
BigDecimal.valueOf(gameData.AI2averageIterations10()).setScale(2, RoundingMode.HALF_EVEN) + "," +
BigDecimal.valueOf(gameData.AI2averageIterations20()).setScale(2, RoundingMode.HALF_EVEN) + "," +
BigDecimal.valueOf(gameData.AI2averageIterations30()).setScale(2, RoundingMode.HALF_EVEN) + "," +
(gameData.nanoAI1() / 1_000_000L) + "," +
(gameData.nanoAI2() / 1_000_000L) + "," +
gameData.time());
writer.newLine();
}
}
public static void writeAIToCsv(String filepath, List<AIData> dataList) throws IOException {
try (BufferedWriter writer = new BufferedWriter(new FileWriter(filepath))) {
writer.write("AI Name,Games Played,Winrate,Average Iterations,Average Iterations 0-10, Average Iterations 11-20, Average Iterations 20-30");
writer.newLine();
for (AIData data : dataList) {
writer.write(
data.getAI() + "," +
data.getGamesPlayed() + "," +
data.getWinrate() + "," +
Math.round(data.getAverageIterations()) + "," +
Math.round(data.getAverageIterations10()) + "," +
Math.round(data.getAverageIterations20()) + "," +
Math.round(data.getAverageIterations30()));
writer.newLine();
}
}
}
}
//public class AITest {
// private static int games = 2;
//
// @BeforeAll
// public static void setUp() {
// var versions = new ArtificialPlayer[5];
// versions[0] = new ArtificialPlayer(new RandomAI(), "Random AI");
// versions[1] = new ArtificialPlayer(new MCTSAI1(20), "MCTS V1 AI");
// versions[2] = new ArtificialPlayer(new org.toop.game.players.ai.mcts.MCTSAI2(20), "MCTS V2 AI");
// versions[3] = new ArtificialPlayer(new org.toop.game.players.ai.mcts.MCTSAI3(20, 10), "MCTS V3 AI");
// versions[4] = new ArtificialPlayer(new MCTSAI4(20, 10), "MCTS V4 AI");
//
// for (int i = 0; i < versions.length; i++) {
// for (int j = i + 1; j < versions.length; j++) {
// final int playerIndex1 = i % versions.length;
// final int playerIndex2 = j % versions.length;
// addMatchup(versions[playerIndex1], versions[playerIndex2]);
// }
// }
//
// }
//
// @BeforeEach
// public void setUpEach() {
// matchupList = new ArrayList<>();
// }
//
// @Test
// public void testIterationsInRealGame() {
// for (int i = 0; i < matchups.size(); i++) {
// testAIVSAI(games, getMatchup(i));
// }
// }
//
//
// private void testAIVSAI(int games, ArtificialPlayer[] ais) {
//
// List<List<Integer>> gamesList = new ArrayList<>();
// for (int i = 0; i < games; i++) {
// final BitboardReversi match = new BitboardReversi();
// match.init(ais);
//
// List<Integer> iterations1 = new ArrayList<>();
// List<Integer> iterations2 = new ArrayList<>();
//
// while (!match.isTerminal()) {
// final int currentAI = match.getCurrentTurn();
// final long move = ais[currentAI].getMove(match);
// if (ais[currentAI].getAi() instanceof MCTSAI) {
// final int lastIterations = ((MCTSAI) ais[currentAI].getAi()).getLastIterations();
// if (currentAI == 0) {
// iterations1.add(lastIterations);
// } else if (currentAI == 1) {
// iterations2.add(lastIterations);
// }
// }
// match.play(move);
// }
// int winner = match.getWinner();
// iterations1.addFirst(winner);
//// iterations1.add(-999);
// iterations1.addAll(iterations2);
//
// gamesList.add(iterations1);
// }
// matchupList.add(gamesList);
// }
//
// @Test
// public void testIterationsAtFixedMove() {
// for (ArtificialPlayer[] matchup : matchups) {
// List<List<Integer>> gamesList = new ArrayList<>();
// for (int j = 0; j < games; j++) {
// final BitboardReversi match = new BitboardReversi();
// match.init(matchup);
//
// List<Integer> iterations = new ArrayList<>();
//
// for (Long move : fixedMoveSet) {
// match.play(move);
// if (move == 32L) {
// break;
// }
// }
//// iterations.add(-999);
// var player = matchup[match.getCurrentTurn()];
// for (int k = 0; k < 10; k++) {
// player.getMove(match);
// if (player.getAi() instanceof MCTSAI) {
// iterations.add(((MCTSAI) player.getAi()).getLastIterations());
// }
// }
// gamesList.add(iterations);
// }
// matchupList.add(gamesList);
// }
// }
//
//
// @Test
// public void testIterationsInFixedGame() {
// for (ArtificialPlayer[] matchup : matchups) {
// List<List<Integer>> gamesList = new ArrayList<>();
// for (int j = 0; j < games; j++) {
// final BitboardReversi match = new BitboardReversi();
// match.init(matchup);
//
// List<Integer> iterations = new ArrayList<>();
//
// iterations.add(-999);
//
// for (Long move : fixedMoveSet) {
// var player = matchup[match.getCurrentTurn()];
// player.getMove(match);
// if (player.getAi() instanceof MCTSAI) {
// iterations.add(((MCTSAI) player.getAi()).getLastIterations());
// }
// match.play(move);
// }
//
// gamesList.add(iterations);
// }
// matchupList.add(gamesList);
// }
// }
//
// @AfterEach
// public void tearDown() {
// data.add(matchupList);
// }
//
// @AfterAll
// public static void writeAfterTests() {
// try {
// writeToCsv("Data.csv", data);
// } catch (IOException e) {
//
// }
// }
//
//
// public static void writeToCsv(String filepath, List<List<List<List<Integer>>>> data) throws IOException {
// try (BufferedWriter writer = new BufferedWriter(new FileWriter(filepath))) {
//
// writer.write("TestID,Matchup,GameNr,Winner");
// for (int i = 0; i < data.size(); i++) {
// writer.write(",Iterations");
// }
//
// writer.newLine();
//
// for (int TestID = 0; TestID < data.size(); TestID++) {
// List<List<List<Integer>>> testCase = data.get(TestID);
//
// for (int matchupNr = 0; matchupNr < testCase.size(); matchupNr++) {
// List<List<Integer>> matchup = testCase.get(matchupNr);
//
// for (int gameNr = 0; gameNr < matchup.size(); gameNr++) {
// List<Integer> game = matchup.get(gameNr);
// writer.write((TestID + 1) + "," + (getMatchupName(matchupNr)) + "," + (gameNr + 1));
// for (int i = 0; i < game.size(); i++) {
// if (i == 0) {
// writer.write("," + getWinnerFromMatchup(game.get(i), matchupNr));
// } else {
// writer.write("," + game.get(i));
// }
// }
// writer.newLine();
// }
// }
// }
// }
//
// }
//
//
// private static final List<List<List<List<Integer>>>> data = new ArrayList<>();
// private List<List<List<Integer>>> matchupList = new ArrayList<>();
// private static final List<String> matchupNames = new ArrayList<>();
// private static final List<ArtificialPlayer[]> matchups = new ArrayList<>();
//
// private static String getMatchupName(int matchupNr) {
// return matchupNames.get(matchupNr);
// }
//
// private static ArtificialPlayer[] getMatchup(int matchupNr) {
// return matchups.get(matchupNr);
// }
//
// private static String getWinnerFromMatchup(Integer winner, int matchupNr) {
// String matchup = matchupNames.get(matchupNr);
//
// String[] parts = matchup.split(" vs ");
//
// if (parts.length != 2) {
// return "Invalid matchup formatting.";
// }
//
// return winner == 0 ? parts[0] : winner == 1 ? parts[1] : winner == -999 ? "NVT" : "Tie";
// }
//
// private static void addMatchup(ArtificialPlayer player1, ArtificialPlayer player2) {
// matchups.add(new ArtificialPlayer[]{player1, player2});
// matchupNames.add(player1.getName() + " vs " + player2.getName());
// }
//}
// private final Long[] fixedMoveSet = new Long[]{17592186044416L,
// 35184372088832L,
// 67108864L,
// 8796093022208L,
// 2251799813685248L,
// 288230376151711744L,
// 70368744177664L,
// 1125899906842624L,
// 137438953472L,
// 140737488355328L,
// 4503599627370496L,
// 2305843009213693952L,
// 18014398509481984L,
// 274877906944L,
// 576460752303423488L,
// -9223372036854775808L,
// 549755813888L,
// 1152921504606846976L,
// 144115188075855872L,
// 72057594037927936L,
// 36028797018963968L,
// 17179869184L,
// 2199023255552L,
// 1048576L,
// 4398046511104L,
// 281474976710656L,
// 9007199254740992L,
// 2147483648L,
// 1073741824L,
// 33554432L,
// 262144L,
// 8388608L,
// 8192L,
// 4611686018427387904L,
// 4294967296L,
// 524288L,
// 4096L,
// 16777216L,
// 65536L,
// 32L,
// 2048L,
// 8L,
// 4L,
// 8589934592L,
// 16L,
// 2097152L,
// 4194304L,
// 1024L,
// 512L,
// 16384L,
// 536870912L,
// 1099511627776L,
// 64L,
// 562949953421312L,
// 128L,
// 1L,
// 32768L,
// 2L,
// 256L,
// 131072L};
// }

View File

@@ -0,0 +1,32 @@
package research;
public record GameData(
String AI1,
String AI2,
String winner,
int turns,
long AI1totalIterations,
long AI1totalIterations10,
long AI1totalIterations20,
long AI1totalIterations30,
double AI1averageIterations,
double AI1averageIterations10,
double AI1averageIterations20,
double AI1averageIterations30,
long AI2totalIterations,
long AI2totalIterations10,
long AI2totalIterations20,
long AI2totalIterations30,
double AI2averageIterations,
double AI2averageIterations10,
double AI2averageIterations20,
double AI2averageIterations30,
long nanoAI1,
long nanoAI2,
String time
) {}

View File

@@ -0,0 +1,30 @@
package research;
import org.toop.framework.game.players.ArtificialPlayer;
import java.util.ArrayList;
import java.util.List;
public class Matchup {
public ArtificialPlayer player1;
public ArtificialPlayer player2;
public Matchup(ArtificialPlayer player1, ArtificialPlayer player2) {
this.player1 = player1;
this.player2 = player2;
}
public Matchup() {}
public String toString() {
return player1.toString() + " VS " + player2.toString();
}
public ArtificialPlayer getPlayer1() {
return player1;
}
public ArtificialPlayer getPlayer2() {
return player2;
}
}